[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:11 dist-test-slave-x4qh krb5kdc[31988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:11 dist-test-slave-x4qh krb5kdc[31988](info): set up 2 sockets
Aug 09 19:30:11 dist-test-slave-x4qh krb5kdc[31988](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 19:30:12.546097 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:12 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:12.655510 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40285
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:40285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:12.735657 32008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:12.735891 32008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:12.735927 32008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:12.737051 32008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:12.737110 32008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:12.737134 32008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:12.737162 32008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:12.737205 32008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:12.738739 32008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:40285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:12.738993 32008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:12.739220 32008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:12.744210 32020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:12.744262 32017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:12.744328 32014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:12.744633 32008 server_base.cc:1034] running on GCE node
I20260809 19:30:12.744889 32008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:12.745219 32008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:12.746383 32008 hybrid_clock.cc:648] HybridClock initialized: now 1786303812746362 us; error 36 us; skew 500 ppm
Aug 09 19:30:12 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303812, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:12.749009 32008 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:12.749475 32008 webserver.cc:460] Webserver started at http://127.31.57.254:36375/ using document root <none> and password file <none>
I20260809 19:30:12.749680 32008 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:12.749720 32008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:12.749833 32008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:12.750988 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "40aee1d978ec44e18468ebb9650c6aa6"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "a90ba615d3388eb1366b1bbc6294d58e"
server_key_iv: "08a97f92ad3c0b169466e4d8802cb983"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.751417 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "40aee1d978ec44e18468ebb9650c6aa6"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "a90ba615d3388eb1366b1bbc6294d58e"
server_key_iv: "08a97f92ad3c0b169466e4d8802cb983"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.752871 32008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:12.753562 32030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:12.753724 32008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:12.753829 32008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "40aee1d978ec44e18468ebb9650c6aa6"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "a90ba615d3388eb1366b1bbc6294d58e"
server_key_iv: "08a97f92ad3c0b169466e4d8802cb983"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.753896 32008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:12.774734 32008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:12.777076 32008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:12.777241 32008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:12.785141 32008 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:40285
I20260809 19:30:12.785197 32116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:40285 every 8 connection(s)
I20260809 19:30:12.785537 32008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:12.786657 32118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:12.788959 32118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6: Bootstrap starting.
I20260809 19:30:12.789513 32118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:12.789755 32118 log.cc:824] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:12.790302 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32008
I20260809 19:30:12.790405 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:12.790529 32118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6: No bootstrap required, opened a new log
I20260809 19:30:12.790997 31975 external_mini_cluster.cc:1428] Setting key 83218c3ff912a49b1c41319648beffa4
I20260809 19:30:12.792448 32118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } }
I20260809 19:30:12.792608 32118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:12.792655 32118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40aee1d978ec44e18468ebb9650c6aa6, State: Initialized, Role: FOLLOWER
I20260809 19:30:12.792779 32118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } }
I20260809 19:30:12.792846 32118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:12.792873 32118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:12.792917 32118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:12.793934 32118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } }
I20260809 19:30:12.794045 32118 leader_election.cc:302] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40aee1d978ec44e18468ebb9650c6aa6; no voters:
I20260809 19:30:12.794214 32118 leader_election.cc:288] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:12.794240 32128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:12.794396 32128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 40aee1d978ec44e18468ebb9650c6aa6, State: Running, Role: LEADER
I20260809 19:30:12.794445 32118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:12.794481 32128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } }
I20260809 19:30:12.794790 32128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40aee1d978ec44e18468ebb9650c6aa6. Latest consensus state: current_term: 1 leader_uuid: "40aee1d978ec44e18468ebb9650c6aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } } }
I20260809 19:30:12.794862 32128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:12.795047 32139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:12.795197 32131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40aee1d978ec44e18468ebb9650c6aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aee1d978ec44e18468ebb9650c6aa6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40285 } } }
I20260809 19:30:12.795248 32131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:12.795588 32139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:12.797737 32139 catalog_manager.cc:1347] Generated new cluster ID: 0e1fad3ca828483996716972bf3208f5
I20260809 19:30:12.797788 32139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:12 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:12.806375 32139 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:12.807190 32139 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:12.809926 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:12.791919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.31.57.254:40285)
0809 19:30:12.792105 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:12.792114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:12.792471 (+ 357us) server_negotiation.cc:407] Connection header received
0809 19:30:12.794897 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:12.794904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:12.794968 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:12.795103 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:12.797354 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.798877 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.800370 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.801126 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.801263 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:12.805427 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:12.805431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:12.805432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:12.805443 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:12.807754 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:12.808258 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:12.808261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:12.808261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:12.808368 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:12.808693 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:12.808694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:12.808694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:12.808912 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:12.809261 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:12.809370 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:30:12.809504 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260809 19:30:12.817396 32139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40aee1d978ec44e18468ebb9650c6aa6: Generated new TSK 0
I20260809 19:30:12.817582 32139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:12.847740 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:12.945817 32166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:12.946008 32166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:12.946030 32166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:12.947073 32166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:12.947108 32166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:12.947134 32166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:12.948463 32166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:12.948644 32166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:12.948807 32166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:12.954452 32174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:12.954427 32171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:12.954493 32166 server_base.cc:1034] running on GCE node
W20260809 19:30:12.954525 32172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:12.954896 32166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:12.955129 32166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:12.956295 32166 hybrid_clock.cc:648] HybridClock initialized: now 1786303812956272 us; error 27 us; skew 500 ppm
Aug 09 19:30:12 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303812, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:12.959095 32166 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:12.959576 32166 webserver.cc:460] Webserver started at http://127.31.57.193:37003/ using document root <none> and password file <none>
I20260809 19:30:12.959795 32166 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:12.959862 32166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:12.960008 32166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:12.961041 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "0f165c0dda434694aad51859dbdec439"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "078f26d04ddeb7d876f9fc44bb3b09b2"
server_key_iv: "81ba2d6a6a80795a102935dfe6e17f07"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.961409 32166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "0f165c0dda434694aad51859dbdec439"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "078f26d04ddeb7d876f9fc44bb3b09b2"
server_key_iv: "81ba2d6a6a80795a102935dfe6e17f07"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.962774 32166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:12.963572 32186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:12.963766 32166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:12.963873 32166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "0f165c0dda434694aad51859dbdec439"
format_stamp: "Formatted at 2026-08-09 19:30:12 on dist-test-slave-x4qh"
server_key: "078f26d04ddeb7d876f9fc44bb3b09b2"
server_key_iv: "81ba2d6a6a80795a102935dfe6e17f07"
server_key_version: "encryptionkey@0"
I20260809 19:30:12.963956 32166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:12.984244 32166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:12.985947 32166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:12.986107 32166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:12.986349 32166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:12.986657 32166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:12.986709 32166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:12.986742 32166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:12.986768 32166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:12.993443 32166 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:38133
I20260809 19:30:12.993480 32382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:38133 every 8 connection(s)
I20260809 19:30:12.993781 32166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:30:12 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303812, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:13.002208 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:12.994608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:39325 (local address 127.31.57.254:40285)
0809 19:30:12.994651 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:12.994653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:12.994856 (+ 203us) server_negotiation.cc:407] Connection header received
0809 19:30:12.995337 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:12.995338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:12.995347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:12.995494 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:12.996411 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.996897 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.997801 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.997987 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.998036 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:12.999953 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:12.999954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:12.999954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:12.999960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:13.001114 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.001472 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.001473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.001473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.001522 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.001723 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.001724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.001724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.001817 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:13.001867 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:13.002078 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 19:30:13.002146 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:13.002362 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:12.994578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40285 (local address 127.31.57.193:39325)
0809 19:30:12.994734 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:12.994774 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:12.995237 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:12.995607 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:12.995611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:12.996140 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:12.996320 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:12.996325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.996977 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.996977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:12.997725 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:12.997727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:12.998057 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:12.998058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:12.998153 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:12.998544 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:12.998551 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:12.999873 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:13.001188 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.001194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.001195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.001408 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.001580 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.001580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.001581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.001670 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.001870 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:13.001874 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:13.001973 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:30:13.002049 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:30:13.002154 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260809 19:30:13.002763 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32166
I20260809 19:30:13.002866 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:13.002910 32383 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40285
I20260809 19:30:13.003003 32383 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:13.003053 31975 external_mini_cluster.cc:1428] Setting key 2da50cfa67f49df25cd3d66e91112398
I20260809 19:30:13.003196 32383 heartbeater.cc:511] Master 127.31.57.254:40285 requested a full tablet report, sending...
I20260809 19:30:13.003986 32058 ts_manager.cc:194] Registered new tserver with Master: 0f165c0dda434694aad51859dbdec439 (127.31.57.193:38133)
I20260809 19:30:13.004913 32058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:39325
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:13.027248 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:13.105103 32388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:13.105268 32388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:13.105289 32388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:13.106319 32388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:13.106359 32388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:13.106382 32388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:13.107647 32388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:13.107805 32388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:13.107946 32388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:13.112407 32397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:13.112419 32401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:13.112447 32394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:13.112710 32388 server_base.cc:1034] running on GCE node
I20260809 19:30:13.112911 32388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:13.113265 32388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:13.114439 32388 hybrid_clock.cc:648] HybridClock initialized: now 1786303813114420 us; error 31 us; skew 500 ppm
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:13.116798 32388 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:13.117251 32388 webserver.cc:460] Webserver started at http://127.31.57.194:39481/ using document root <none> and password file <none>
I20260809 19:30:13.117472 32388 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:13.117535 32388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:13.117673 32388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:13.118562 32388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "7fd50e9f2efa445d9cb14c03aa4f8d08"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "9246326852b6dc7bd58f11016bfad2f4"
server_key_iv: "1527e6dde3ff3d2448dd81ecd4508e7e"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.118892 32388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "7fd50e9f2efa445d9cb14c03aa4f8d08"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "9246326852b6dc7bd58f11016bfad2f4"
server_key_iv: "1527e6dde3ff3d2448dd81ecd4508e7e"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.120265 32388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:13.120944 32411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.121129 32388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:13.121232 32388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "7fd50e9f2efa445d9cb14c03aa4f8d08"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "9246326852b6dc7bd58f11016bfad2f4"
server_key_iv: "1527e6dde3ff3d2448dd81ecd4508e7e"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.121315 32388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:13.145560 32388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:13.147663 32388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:13.147818 32388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:13.148022 32388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:13.148305 32388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:13.148355 32388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.148386 32388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:13.148412 32388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.155279 32388 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:41611
I20260809 19:30:13.155313 32612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:41611 every 8 connection(s)
I20260809 19:30:13.155601 32388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:13.161852 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32388
I20260809 19:30:13.161953 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:13.162137 31975 external_mini_cluster.cc:1428] Setting key b86c1842789cf651ffa53b2b41d0f8de
I20260809 19:30:13.163998 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.156229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:53201 (local address 127.31.57.254:40285)
0809 19:30:13.156266 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:13.156268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:13.156437 (+ 169us) server_negotiation.cc:407] Connection header received
0809 19:30:13.156839 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:13.156839 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:13.156847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:13.156897 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:13.157781 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.158219 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.159048 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.159218 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.159265 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.161054 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:13.161055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:13.161056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:13.161068 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:13.162706 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.163100 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.163102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.163102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.163161 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.163421 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.163422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.163422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.163525 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:13.163601 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:13.163848 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 19:30:13.163919 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:30:13.164355 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.156184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40285 (local address 127.31.57.194:53201)
0809 19:30:13.156328 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:13.156364 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:13.156757 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:13.156950 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:13.156952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:13.157472 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:13.157692 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:13.157696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.158290 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.158290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:13.158977 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:13.158978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.159284 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.159285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:13.159415 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.159809 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:13.159816 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:13.160962 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:13.162792 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.162799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.162801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.163025 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.163226 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.163227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.163228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.163332 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.163594 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:13.163601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:13.163725 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 19:30:13.163817 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 19:30:13.163949 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":107,"threads_started":1}
I20260809 19:30:13.164987 32613 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40285
I20260809 19:30:13.165078 32613 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:13.165239 32613 heartbeater.cc:511] Master 127.31.57.254:40285 requested a full tablet report, sending...
I20260809 19:30:13.165630 32058 ts_manager.cc:194] Registered new tserver with Master: 7fd50e9f2efa445d9cb14c03aa4f8d08 (127.31.57.194:41611)
I20260809 19:30:13.166141 32058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:53201
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:13.186436 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:13.262609 32623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:13.262799 32623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:13.262825 32623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:13.264014 32623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:13.264058 32623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:13.264086 32623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:13.265468 32623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:13.265669 32623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:13.265830 32623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:13.270946 32630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:13.270988 32629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:13.272541 32633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:13.272539 32623 server_base.cc:1034] running on GCE node
I20260809 19:30:13.272789 32623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:13.272974 32623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:13.274129 32623 hybrid_clock.cc:648] HybridClock initialized: now 1786303813274112 us; error 29 us; skew 500 ppm
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:13.276762 32623 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:13.277150 32623 webserver.cc:460] Webserver started at http://127.31.57.195:34987/ using document root <none> and password file <none>
I20260809 19:30:13.277343 32623 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:13.277382 32623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:13.277472 32623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:13.278371 32623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "534fd7d4f252467981b6beb3ccffbefe"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "cf7567b9fdf77c42d5c8655aab06cf4c"
server_key_iv: "e255ff1d242f87f392c3717da0bb4824"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.278649 32623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "534fd7d4f252467981b6beb3ccffbefe"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "cf7567b9fdf77c42d5c8655aab06cf4c"
server_key_iv: "e255ff1d242f87f392c3717da0bb4824"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.279973 32623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:13.280632 32645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.280838 32623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:13.280928 32623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "534fd7d4f252467981b6beb3ccffbefe"
format_stamp: "Formatted at 2026-08-09 19:30:13 on dist-test-slave-x4qh"
server_key: "cf7567b9fdf77c42d5c8655aab06cf4c"
server_key_iv: "e255ff1d242f87f392c3717da0bb4824"
server_key_version: "encryptionkey@0"
I20260809 19:30:13.281000 32623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:13.319038 32623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:13.320767 32623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:13.320960 32623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:13.321177 32623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:13.321501 32623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:13.321535 32623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.321556 32623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:13.321614 32623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:13.327709 32623 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:37815
I20260809 19:30:13.327802 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:37815 every 8 connection(s)
I20260809 19:30:13.328094 32623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:13.330888 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32623
I20260809 19:30:13.330989 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:13.331125 31975 external_mini_cluster.cc:1428] Setting key e55f4d93d7dd5668ffe24f70812ce566
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:13.337463 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.328808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:43321 (local address 127.31.57.254:40285)
0809 19:30:13.328852 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:13.328854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:13.329015 (+ 161us) server_negotiation.cc:407] Connection header received
0809 19:30:13.329488 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:13.329489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:13.329501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:13.329567 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:13.330379 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.330828 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.332235 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.332458 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.332541 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.334742 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:13.334744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:13.334744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:13.334752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:13.335982 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.336448 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.336449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.336449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.336515 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.336802 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.336804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.336804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.336924 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:13.337010 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:13.337285 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 19:30:13.337371 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:13.337648 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.328785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40285 (local address 127.31.57.195:43321)
0809 19:30:13.328921 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:13.328952 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:13.329380 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:13.329617 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:13.329620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:13.330130 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:13.330305 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:13.330309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.330951 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.330952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:13.332152 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:13.332155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.332552 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.332553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:13.332698 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.333109 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:13.333116 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:13.334617 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:13.336089 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.336096 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.336099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.336348 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.336605 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:13.336605 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:13.336606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:13.336725 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:13.336999 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:13.337005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:13.337146 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 19:30:13.337260 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:30:13.337410 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260809 19:30:13.338176 380 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40285
I20260809 19:30:13.338302 380 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:13.338495 380 heartbeater.cc:511] Master 127.31.57.254:40285 requested a full tablet report, sending...
I20260809 19:30:13.338853 32058 ts_manager.cc:194] Registered new tserver with Master: 534fd7d4f252467981b6beb3ccffbefe (127.31.57.195:37815)
I20260809 19:30:13.339200 32058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:43321
I20260809 19:30:13.343153 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:13.365898 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.358744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.31.57.193:38133)
0809 19:30:13.358891 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:13.358899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:13.358908 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:13.358931 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:13.358932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:13.358944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:13.359007 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:13.359650 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.360324 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.360970 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.361247 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.361313 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.363347 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:13.363353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:13.363353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:13.363359 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:13.364957 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.365265 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.365267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.365267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.365318 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.365479 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.365479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.365480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.365589 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:13.365691 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:13.365756 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:30:13.365829 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
W20260809 19:30:13.366264 32228 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33464
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[31988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:13.387192 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:13.380416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.31.57.193:38133)
0809 19:30:13.380485 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:13.380486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:13.380579 (+ 93us) server_negotiation.cc:407] Connection header received
0809 19:30:13.380636 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:13.380637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:13.380644 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:13.380692 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:13.381220 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.381661 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.382348 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:13.382534 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:13.382578 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:13.384883 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:13.384885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:13.384885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:13.384891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:13.386162 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.386494 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.386495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.386495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.386548 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:13.386742 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:13.386743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:13.386743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:13.386835 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:13.386939 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:13.387035 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:30:13.387102 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:30:13.387687 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32166
I20260809 19:30:13.403167 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32388
I20260809 19:30:13.409920 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32623
I20260809 19:30:13.419332 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32008
2026-08-09T19:30:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2083 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[404](info): set up 2 sockets
Aug 09 19:30:13 dist-test-slave-x4qh krb5kdc[404](info): commencing operation
krb5kdc: starting...
W20260809 19:30:14.585232 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:14 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:14.695924 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39531
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:14.772526 426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:14.772707 426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:14.772728 426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:14.773813 426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:14.773850 426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:14.773864 426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:14.773912 426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:14.773928 426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:14.775314 426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:14.775578 426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:14.775830 426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:14.780342 439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:14.780355 440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:14.780469 426 server_base.cc:1034] running on GCE node
W20260809 19:30:14.780355 443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:14.780772 426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:14.781047 426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:14.782191 426 hybrid_clock.cc:648] HybridClock initialized: now 1786303814782172 us; error 35 us; skew 500 ppm
Aug 09 19:30:14 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:14.784688 426 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:14.785041 426 webserver.cc:460] Webserver started at http://127.31.57.254:35381/ using document root <none> and password file <none>
I20260809 19:30:14.785230 426 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:14.785269 426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:14.785360 426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:14.786249 426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "f3beedf5f59542c5ba69acf7902e0280"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "893d740ef6ee50445c8677dbce823873"
server_key_iv: "af8ad1c6ee7d4bac03d9a26171f3d7de"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.786537 426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "f3beedf5f59542c5ba69acf7902e0280"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "893d740ef6ee50445c8677dbce823873"
server_key_iv: "af8ad1c6ee7d4bac03d9a26171f3d7de"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.787812 426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:14.788482 455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:14.788654 426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:14.788755 426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "f3beedf5f59542c5ba69acf7902e0280"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "893d740ef6ee50445c8677dbce823873"
server_key_iv: "af8ad1c6ee7d4bac03d9a26171f3d7de"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.788825 426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:14.807233 426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:14.808916 426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:14.809106 426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:14.812786 426 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:39531
I20260809 19:30:14.812820 534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:39531 every 8 connection(s)
I20260809 19:30:14.813126 426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:14.813896 536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:14.815954 536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280: Bootstrap starting.
I20260809 19:30:14.816495 536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:14.816771 536 log.cc:824] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:14.817414 536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280: No bootstrap required, opened a new log
I20260809 19:30:14.818820 536 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } }
I20260809 19:30:14.818939 536 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:14.818992 536 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3beedf5f59542c5ba69acf7902e0280, State: Initialized, Role: FOLLOWER
I20260809 19:30:14.819128 536 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } }
I20260809 19:30:14.819206 536 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:14.819255 536 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:14.819306 536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:14.819903 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 426
I20260809 19:30:14.819948 536 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } }
I20260809 19:30:14.819988 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:14.820050 536 leader_election.cc:302] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3beedf5f59542c5ba69acf7902e0280; no voters:
I20260809 19:30:14.820137 31975 external_mini_cluster.cc:1428] Setting key a3175e24dcc47a6e76ac5df1e4a81259
I20260809 19:30:14.820204 536 leader_election.cc:288] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:14.820243 542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:14.820402 542 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [term 1 LEADER]: Becoming Leader. State: Replica: f3beedf5f59542c5ba69acf7902e0280, State: Running, Role: LEADER
I20260809 19:30:14.820444 536 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:14.820488 542 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } }
I20260809 19:30:14.820812 542 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3beedf5f59542c5ba69acf7902e0280. Latest consensus state: current_term: 1 leader_uuid: "f3beedf5f59542c5ba69acf7902e0280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } } }
I20260809 19:30:14.820899 542 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:14.820955 543 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3beedf5f59542c5ba69acf7902e0280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3beedf5f59542c5ba69acf7902e0280" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39531 } } }
I20260809 19:30:14.821049 543 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:14.821121 559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:14.821630 559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:14.823191 559 catalog_manager.cc:1347] Generated new cluster ID: e837c7ad6d2f428a8fb26886e9fe7b54
I20260809 19:30:14.823241 559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:14 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:14.830753 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:14.820577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.31.57.254:39531)
0809 19:30:14.821145 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:14.821154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:14.821186 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:30:14.821293 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:14.821301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:14.821357 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:14.822277 (+ 920us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:14.822347 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.823505 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.824476 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.824852 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.824922 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:14.827477 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:14.827481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:14.827481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:14.827492 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:14.829208 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:14.829595 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:14.829598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:14.829598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:14.829701 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:14.829905 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:14.829906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:14.829906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:14.830166 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:14.830302 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:14.830469 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 19:30:14.830587 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":66,"threads_started":1}
I20260809 19:30:14.837409 559 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:14.838032 559 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:14.843760 559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3beedf5f59542c5ba69acf7902e0280: Generated new TSK 0
I20260809 19:30:14.843919 559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:14.871888 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:14.948494 583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:14.948704 583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:14.948740 583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:14.949896 583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:14.949958 583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:14.950002 583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:14.951541 583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:14.951803 583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:14.952023 583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:14.957113 592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:14.957114 597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:14.957203 593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:14.957300 583 server_base.cc:1034] running on GCE node
I20260809 19:30:14.957579 583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:14.957774 583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:14.958912 583 hybrid_clock.cc:648] HybridClock initialized: now 1786303814958902 us; error 28 us; skew 500 ppm
Aug 09 19:30:14 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:14.961452 583 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:14.961869 583 webserver.cc:460] Webserver started at http://127.31.57.193:42985/ using document root <none> and password file <none>
I20260809 19:30:14.962136 583 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:14.962180 583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:14.962352 583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:14.963438 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "b1af6849c41e4575ace9f6229af85421"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "52af885290215149ee17cac7d7421ae5"
server_key_iv: "cd9b5403f53aee84359a226f89a717f8"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.963815 583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "b1af6849c41e4575ace9f6229af85421"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "52af885290215149ee17cac7d7421ae5"
server_key_iv: "cd9b5403f53aee84359a226f89a717f8"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.965204 583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:14.965919 612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:14.966113 583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:14.966209 583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "b1af6849c41e4575ace9f6229af85421"
format_stamp: "Formatted at 2026-08-09 19:30:14 on dist-test-slave-x4qh"
server_key: "52af885290215149ee17cac7d7421ae5"
server_key_iv: "cd9b5403f53aee84359a226f89a717f8"
server_key_version: "encryptionkey@0"
I20260809 19:30:14.966275 583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:14.983783 583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:14.986091 583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:14.986255 583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:14.986483 583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:14.986780 583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:14.986820 583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:14.986862 583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:14.986887 583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:14.993016 583 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:44421
I20260809 19:30:14.993098 799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:44421 every 8 connection(s)
I20260809 19:30:14.993374 583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:14.996572 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 583
I20260809 19:30:14.996666 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:14.996819 31975 external_mini_cluster.cc:1428] Setting key 7885a278ba0b7b63c43de0edfd6830cf
Aug 09 19:30:14 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:15.002595 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:14.994082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:47309 (local address 127.31.57.254:39531)
0809 19:30:14.994140 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:14.994141 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:14.994345 (+ 204us) server_negotiation.cc:407] Connection header received
0809 19:30:14.995196 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:14.995196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:14.995210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:14.995264 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:14.995897 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.996380 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.997458 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.997672 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.997757 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:14.999907 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:14.999908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:14.999909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:14.999915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.001221 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.001653 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.001655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.001655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.001714 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.001973 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.001973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.001974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.002086 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.002163 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.002426 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.002509 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:30:15.002775 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:14.994068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39531 (local address 127.31.57.193:47309)
0809 19:30:14.994228 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:14.994265 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:14.995106 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:14.995315 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:14.995318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:14.995632 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:14.995822 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:14.995827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.996480 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.996480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:14.997361 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:14.997363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:14.997771 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:14.997772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:14.997922 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:14.998436 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:14.998444 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:14.999809 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:15.001325 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.001332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.001334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.001561 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.001783 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.001783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.001784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.001897 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.002154 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:15.002161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:15.002295 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.002403 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.002550 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260809 19:30:15.003242 800 heartbeater.cc:347] Connected to a master server at 127.31.57.254:39531
I20260809 19:30:15.003361 800 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:15.003536 800 heartbeater.cc:511] Master 127.31.57.254:39531 requested a full tablet report, sending...
I20260809 19:30:15.004057 477 ts_manager.cc:194] Registered new tserver with Master: b1af6849c41e4575ace9f6229af85421 (127.31.57.193:44421)
I20260809 19:30:15.005014 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:47309
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:15.021970 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:15.098249 807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:15.098423 807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:15.098443 807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:15.099416 807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:15.099462 807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:15.099486 807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:15.100704 807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:15.100850 807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:15.100989 807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:15.105490 817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:15.105521 816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:15.105593 807 server_base.cc:1034] running on GCE node
W20260809 19:30:15.105669 819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:15.105876 807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:15.106101 807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:15.107235 807 hybrid_clock.cc:648] HybridClock initialized: now 1786303815107224 us; error 27 us; skew 500 ppm
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:15.109443 807 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:15.109874 807 webserver.cc:460] Webserver started at http://127.31.57.194:40745/ using document root <none> and password file <none>
I20260809 19:30:15.110077 807 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:15.110133 807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:15.110250 807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:15.111162 807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "d0e3858d3e464011bca1a7d513d1dc2d"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "1d3319ec592b7434b9b173640bf09ae7"
server_key_iv: "9f6d91ae596377108626f3e659636c2a"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.111542 807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "d0e3858d3e464011bca1a7d513d1dc2d"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "1d3319ec592b7434b9b173640bf09ae7"
server_key_iv: "9f6d91ae596377108626f3e659636c2a"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.112802 807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:15.113761 835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.113950 807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:15.114048 807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "d0e3858d3e464011bca1a7d513d1dc2d"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "1d3319ec592b7434b9b173640bf09ae7"
server_key_iv: "9f6d91ae596377108626f3e659636c2a"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.114130 807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:15.138821 807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:15.140475 807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:15.140583 807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:15.140744 807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:15.141036 807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:15.141096 807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.141139 807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:15.141168 807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.147042 807 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:41815
I20260809 19:30:15.147079 1013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:41815 every 8 connection(s)
I20260809 19:30:15.147420 807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:15.155755 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.148121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:57657 (local address 127.31.57.254:39531)
0809 19:30:15.148198 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.148200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.148359 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:30:15.149198 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.149199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.149208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.149251 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.149765 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.150207 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.151052 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.151216 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.151260 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.153222 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.153224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.153224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.153231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.154581 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.154935 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.154936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.154936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.154982 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.155189 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.155190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.155190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.155277 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.155322 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.155638 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.155700 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:30:15.155965 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.148073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39531 (local address 127.31.57.194:57657)
0809 19:30:15.148244 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.148277 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.149113 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.149306 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.149309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.149531 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:15.149699 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.149703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.150282 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.150283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.150982 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.150984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.151287 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.151287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.151458 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.151832 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:15.151839 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:15.153136 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:15.154650 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.154655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.154657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.154869 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.155039 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.155039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.155040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.155136 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.155360 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:15.155366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:15.155529 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.155617 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.155743 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260809 19:30:15.156471 1014 heartbeater.cc:347] Connected to a master server at 127.31.57.254:39531
I20260809 19:30:15.156554 1014 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:15.156708 1014 heartbeater.cc:511] Master 127.31.57.254:39531 requested a full tablet report, sending...
I20260809 19:30:15.156714 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 807
I20260809 19:30:15.156792 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:15.156978 31975 external_mini_cluster.cc:1428] Setting key 371933c673015e1e939b594e21dab0cd
I20260809 19:30:15.157104 477 ts_manager.cc:194] Registered new tserver with Master: d0e3858d3e464011bca1a7d513d1dc2d (127.31.57.194:41815)
I20260809 19:30:15.157451 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:57657
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:15.181427 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:15.260087 1023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:15.260288 1023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:15.260313 1023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:15.261354 1023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:15.261391 1023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:15.261421 1023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:15.262738 1023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:39531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:15.262904 1023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:15.263062 1023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:15.267853 1029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:15.267856 1036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:15.267867 1030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:15.268316 1023 server_base.cc:1034] running on GCE node
I20260809 19:30:15.268545 1023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:15.268752 1023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:15.269892 1023 hybrid_clock.cc:648] HybridClock initialized: now 1786303815269849 us; error 58 us; skew 500 ppm
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:15.272497 1023 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:15.272985 1023 webserver.cc:460] Webserver started at http://127.31.57.195:44161/ using document root <none> and password file <none>
I20260809 19:30:15.273224 1023 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:15.273268 1023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:15.273412 1023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:15.274437 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "d482646f2f924c9ea7c8855fab7dc685"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "3fa4cb8f70e08c19015f518a8da20bf7"
server_key_iv: "8260a9cb0af09f530677a28090166c70"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.274838 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "d482646f2f924c9ea7c8855fab7dc685"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "3fa4cb8f70e08c19015f518a8da20bf7"
server_key_iv: "8260a9cb0af09f530677a28090166c70"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.276821 1023 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 19:30:15.277452 1049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.277628 1023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.277706 1023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "d482646f2f924c9ea7c8855fab7dc685"
format_stamp: "Formatted at 2026-08-09 19:30:15 on dist-test-slave-x4qh"
server_key: "3fa4cb8f70e08c19015f518a8da20bf7"
server_key_iv: "8260a9cb0af09f530677a28090166c70"
server_key_version: "encryptionkey@0"
I20260809 19:30:15.277755 1023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:15.290988 1023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:15.293167 1023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:15.293274 1023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:15.293462 1023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:15.293681 1023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:15.293707 1023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.293725 1023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:15.293737 1023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:15.299146 1023 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:46805
I20260809 19:30:15.299218 1247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:46805 every 8 connection(s)
I20260809 19:30:15.299546 1023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:15.306385 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1023
I20260809 19:30:15.306496 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:15.306675 31975 external_mini_cluster.cc:1428] Setting key 158ee1a55acaa6332b757ba0a78821dd
I20260809 19:30:15.308710 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.300322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:60917 (local address 127.31.57.254:39531)
0809 19:30:15.300385 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.300387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.300646 (+ 259us) server_negotiation.cc:407] Connection header received
0809 19:30:15.301577 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.301578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.301587 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.301639 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.302221 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.302737 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.303763 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.303944 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.303997 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.305967 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.305968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.305968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.305975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.307613 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.307969 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.307970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.307971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.308025 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.308210 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.308211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.308211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.308313 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.308371 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.308559 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.308652 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:15.308764 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.300293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39531 (local address 127.31.57.195:60917)
0809 19:30:15.300474 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.300514 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.301477 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.301705 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.301708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.301949 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:15.302132 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.302136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.302820 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.302821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.303686 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.303689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.304017 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.304018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.304114 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.304529 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:15.304536 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:15.305881 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:15.307687 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.307693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.307695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.307906 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.308079 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:15.308079 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:15.308080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:15.308159 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:15.308362 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:15.308365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:15.308460 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.308527 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.308629 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260809 19:30:15.309235 1248 heartbeater.cc:347] Connected to a master server at 127.31.57.254:39531
I20260809 19:30:15.309315 1248 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:15.309487 1248 heartbeater.cc:511] Master 127.31.57.254:39531 requested a full tablet report, sending...
I20260809 19:30:15.309881 477 ts_manager.cc:194] Registered new tserver with Master: d482646f2f924c9ea7c8855fab7dc685 (127.31.57.195:46805)
I20260809 19:30:15.310290 477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:60917
I20260809 19:30:15.318486 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:30:15.325780 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.320094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.31.57.254:39531)
0809 19:30:15.320128 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.320130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.320260 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:30:15.320329 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.320330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.320339 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.320378 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.320893 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.321437 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.322250 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.322447 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.322529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.323591 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.323593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.323593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.323599 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.324715 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.325035 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.325036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.325036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.325091 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.325280 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.325281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.325281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.325410 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.325506 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.325613 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.325696 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:30:15.327591 477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:15.328975 477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:15.335016 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.330979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:46805 (local address 127.0.0.1:58522)
0809 19:30:15.331505 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.331537 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.331621 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.331896 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.331897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.331900 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:15.331965 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.331967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.332862 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.332863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.333852 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.333855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.334692 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.334693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.334764 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.334776 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.334890 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.334953 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":47,"threads_started":1}
I20260809 19:30:15.335031 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.331249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58522 (local address 127.31.57.195:46805)
0809 19:30:15.331475 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.331479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.331661 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:30:15.331705 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.331708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.331722 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.331793 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:15.332049 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.332766 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.333941 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.334609 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.334693 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.334746 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.334920 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.334955 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
I20260809 19:30:15.335335 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.330954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:44421 (local address 127.0.0.1:46324)
0809 19:30:15.331498 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.331536 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.331643 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.331876 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.331880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.331882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:15.331990 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.331993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.333275 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.333277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.334171 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.334173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.335066 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.335067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.335141 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.335146 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.335221 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.335257 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":24,"thread_start_us":47,"threads_started":1}
I20260809 19:30:15.335361 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.331059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.31.57.193:44421)
0809 19:30:15.331228 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.331232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.331632 (+ 400us) server_negotiation.cc:407] Connection header received
0809 19:30:15.331756 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.331761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.331778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.331854 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:15.332085 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.333156 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.334250 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.334918 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.334983 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.335031 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.335244 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.335293 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260809 19:30:15.335651 1143 tablet_service.cc:1467] Processing CreateTablet for tablet fdcdcb730e80465099cd96ae09f348c0 (DEFAULT_TABLE table=test-table [id=54a285877b19405c819a8385f83755b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:15.335990 1143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcdcb730e80465099cd96ae09f348c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:15.336012 1264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.331147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:41815 (local address 127.0.0.1:42436)
0809 19:30:15.331720 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.331729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.331776 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.332136 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.332137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.332139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:15.332179 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.332180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.333364 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.333365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.334389 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.334392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.335780 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.335781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.335843 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.335846 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.335901 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.335943 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":39,"threads_started":1}
I20260809 19:30:15.336306 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.331252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.31.57.194:41815)
0809 19:30:15.331491 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.331496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.331785 (+ 289us) server_negotiation.cc:407] Connection header received
0809 19:30:15.331952 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.331955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.331970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.332042 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:15.332265 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.333243 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.334489 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.335700 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.335753 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.335785 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.336192 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.336238 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":118,"threads_started":1}
I20260809 19:30:15.336390 697 tablet_service.cc:1467] Processing CreateTablet for tablet fdcdcb730e80465099cd96ae09f348c0 (DEFAULT_TABLE table=test-table [id=54a285877b19405c819a8385f83755b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:15.336727 697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcdcb730e80465099cd96ae09f348c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:15.338606 1271 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Bootstrap starting.
I20260809 19:30:15.339288 1272 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Bootstrap starting.
I20260809 19:30:15.339327 1271 tablet_bootstrap.cc:654] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:15.339823 1272 tablet_bootstrap.cc:654] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:15.339880 1271 log.cc:824] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:15.339826 906 tablet_service.cc:1467] Processing CreateTablet for tablet fdcdcb730e80465099cd96ae09f348c0 (DEFAULT_TABLE table=test-table [id=54a285877b19405c819a8385f83755b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:15.340135 906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcdcb730e80465099cd96ae09f348c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:15.340382 1272 log.cc:824] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:15.340646 1271 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: No bootstrap required, opened a new log
I20260809 19:30:15.340718 1271 ts_tablet_manager.cc:1397] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:15.341053 1272 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: No bootstrap required, opened a new log
I20260809 19:30:15.341141 1272 ts_tablet_manager.cc:1397] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:15.342657 1271 raft_consensus.cc:348] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.342834 1271 raft_consensus.cc:374] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:15.342732 1272 raft_consensus.cc:348] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.342886 1271 raft_consensus.cc:729] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d482646f2f924c9ea7c8855fab7dc685, State: Initialized, Role: FOLLOWER
I20260809 19:30:15.342886 1272 raft_consensus.cc:374] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:15.342921 1272 raft_consensus.cc:729] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1af6849c41e4575ace9f6229af85421, State: Initialized, Role: FOLLOWER
I20260809 19:30:15.343006 1271 consensus_queue.cc:260] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.343053 1272 consensus_queue.cc:260] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.343253 1271 ts_tablet_manager.cc:1428] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:30:15.343330 1272 ts_tablet_manager.cc:1428] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:15.343335 1282 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Bootstrap starting.
I20260809 19:30:15.343400 1248 heartbeater.cc:503] Master 127.31.57.254:39531 was elected leader, sending a full tablet report...
I20260809 19:30:15.343492 800 heartbeater.cc:503] Master 127.31.57.254:39531 was elected leader, sending a full tablet report...
I20260809 19:30:15.343924 1282 tablet_bootstrap.cc:654] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:15.344528 1282 log.cc:824] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:15.345321 1282 tablet_bootstrap.cc:492] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: No bootstrap required, opened a new log
I20260809 19:30:15.345381 1282 ts_tablet_manager.cc:1397] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:15.346539 1282 raft_consensus.cc:348] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.346704 1282 raft_consensus.cc:374] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:15.346769 1282 raft_consensus.cc:729] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e3858d3e464011bca1a7d513d1dc2d, State: Initialized, Role: FOLLOWER
I20260809 19:30:15.346889 1282 consensus_queue.cc:260] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.347082 1282 ts_tablet_manager.cc:1428] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:15.347146 1014 heartbeater.cc:503] Master 127.31.57.254:39531 was elected leader, sending a full tablet report...
W20260809 19:30:15.398043 1017 tablet.cc:2369] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:15.494167 802 tablet.cc:2369] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:15.550122 1249 tablet.cc:2369] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:15.734285 1289 raft_consensus.cc:482] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:15.734426 1289 raft_consensus.cc:504] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.734843 1289 leader_election.cc:288] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1af6849c41e4575ace9f6229af85421 (127.31.57.193:44421), d482646f2f924c9ea7c8855fab7dc685 (127.31.57.195:46805)
I20260809 19:30:15.738799 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.735103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:60101 (local address 127.31.57.193:44421)
0809 19:30:15.735152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.735154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.735400 (+ 246us) server_negotiation.cc:407] Connection header received
0809 19:30:15.735424 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.735425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.735433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.735470 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:15.735663 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.736437 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.737940 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.738453 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.738524 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.738532 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.738696 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.738733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:15.738799 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.735186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:38813 (local address 127.31.57.195:46805)
0809 19:30:15.735236 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.735238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.735276 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:30:15.735300 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.735301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.735307 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.735349 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:15.735664 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.736427 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.737939 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.738453 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.738525 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.738532 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.738696 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.738733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:30:15.738886 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.735030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:44421 (local address 127.31.57.194:60101)
0809 19:30:15.735147 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.735158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.735214 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.735494 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.735495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.735496 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:15.735564 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.735566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.736520 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.736521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.737846 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.737848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.738542 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.738542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.738613 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.738617 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.738673 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.738725 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":79,"thread_start_us":60,"threads_started":1}
I20260809 19:30:15.738890 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.735030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:46805 (local address 127.31.57.194:38813)
0809 19:30:15.735148 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:15.735158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:15.735212 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:15.735401 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:15.735402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:15.735405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:15.735504 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.735506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.736521 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.736522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.737845 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:15.737848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.738539 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.738540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:15.738615 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.738617 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:15.738673 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:30:15.738725 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":97,"thread_start_us":56,"threads_started":1}
I20260809 19:30:15.739348 728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcdcb730e80465099cd96ae09f348c0" candidate_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1af6849c41e4575ace9f6229af85421" is_pre_election: true
I20260809 19:30:15.739424 1175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcdcb730e80465099cd96ae09f348c0" candidate_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d482646f2f924c9ea7c8855fab7dc685" is_pre_election: true
I20260809 19:30:15.739549 728 raft_consensus.cc:2454] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e3858d3e464011bca1a7d513d1dc2d in term 0.
I20260809 19:30:15.739562 1175 raft_consensus.cc:2454] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0e3858d3e464011bca1a7d513d1dc2d in term 0.
I20260809 19:30:15.739771 840 leader_election.cc:302] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0e3858d3e464011bca1a7d513d1dc2d, d482646f2f924c9ea7c8855fab7dc685; no voters:
I20260809 19:30:15.739898 1289 raft_consensus.cc:2781] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:15.739950 1289 raft_consensus.cc:482] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:15.739982 1289 raft_consensus.cc:3037] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:15.740641 1289 raft_consensus.cc:504] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.740759 1289 leader_election.cc:288] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [CANDIDATE]: Term 1 election: Requested vote from peers b1af6849c41e4575ace9f6229af85421 (127.31.57.193:44421), d482646f2f924c9ea7c8855fab7dc685 (127.31.57.195:46805)
I20260809 19:30:15.740931 1175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcdcb730e80465099cd96ae09f348c0" candidate_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d482646f2f924c9ea7c8855fab7dc685"
I20260809 19:30:15.740955 728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcdcb730e80465099cd96ae09f348c0" candidate_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1af6849c41e4575ace9f6229af85421"
I20260809 19:30:15.741012 1175 raft_consensus.cc:3037] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:15.741022 728 raft_consensus.cc:3037] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:15.741704 1175 raft_consensus.cc:2454] T fdcdcb730e80465099cd96ae09f348c0 P d482646f2f924c9ea7c8855fab7dc685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e3858d3e464011bca1a7d513d1dc2d in term 1.
I20260809 19:30:15.741909 840 leader_election.cc:302] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0e3858d3e464011bca1a7d513d1dc2d, d482646f2f924c9ea7c8855fab7dc685; no voters:
I20260809 19:30:15.741914 728 raft_consensus.cc:2454] T fdcdcb730e80465099cd96ae09f348c0 P b1af6849c41e4575ace9f6229af85421 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e3858d3e464011bca1a7d513d1dc2d in term 1.
I20260809 19:30:15.742048 1289 raft_consensus.cc:2781] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:15.742268 1289 raft_consensus.cc:686] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [term 1 LEADER]: Becoming Leader. State: Replica: d0e3858d3e464011bca1a7d513d1dc2d, State: Running, Role: LEADER
I20260809 19:30:15.742374 1289 consensus_queue.cc:237] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } }
I20260809 19:30:15.743269 476 catalog_manager.cc:5495] T fdcdcb730e80465099cd96ae09f348c0 P d0e3858d3e464011bca1a7d513d1dc2d reported cstate change: term changed from 0 to 1, leader changed from <none> to d0e3858d3e464011bca1a7d513d1dc2d (127.31.57.194). New cstate: current_term: 1 leader_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1af6849c41e4575ace9f6229af85421" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d482646f2f924c9ea7c8855fab7dc685" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0e3858d3e464011bca1a7d513d1dc2d" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41815 } health_report { overall_health: HEALTHY } } }
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:15.776541 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.769888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.31.57.193:44421)
0809 19:30:15.769962 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.769963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.770061 (+ 98us) server_negotiation.cc:407] Connection header received
0809 19:30:15.770130 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.770131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.770146 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.770205 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.770725 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.771181 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.771894 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.772078 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.772171 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.774231 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.774240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.774240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.774245 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.775600 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.775912 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.775914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.775914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.775972 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.776159 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.776160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.776160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.776276 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.776372 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.776433 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.776487 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:15.800074 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.793221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.31.57.193:44421)
0809 19:30:15.793286 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.793288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.793398 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:30:15.793488 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.793489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.793499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.793620 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.794152 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.794589 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.795275 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.795563 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.795642 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.797646 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.797648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.797648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.797655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.798842 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.799220 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.799221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.799221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.799278 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.799529 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.799530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.799531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.799636 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.799740 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.799905 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.799982 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:15.821952 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:15.815348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.31.57.193:44421)
0809 19:30:15.815457 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:15.815459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:15.815536 (+ 77us) server_negotiation.cc:407] Connection header received
0809 19:30:15.815573 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:15.815574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:15.815582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:15.815653 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:15.816147 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.816661 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.817339 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:15.817497 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:15.817550 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:15.819734 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:15.819735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:15.819736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:15.819741 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:15.820955 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.821267 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.821268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.821268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.821346 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:15.821553 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:15.821554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:15.821554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:15.821665 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:15.821742 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:15.821841 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:30:15.821897 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 19:30:15.822399 662 tablet_service.cc:2908] Rejecting scan request for tablet fdcdcb730e80465099cd96ae09f348c0: Uninitialized: clean time has not yet been initialized
I20260809 19:30:15.823275 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 583
I20260809 19:30:15.839208 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 807
I20260809 19:30:15.845508 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1023
I20260809 19:30:15.852247 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 426
2026-08-09T19:30:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2434 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[1319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[1319](info): set up 2 sockets
Aug 09 19:30:15 dist-test-slave-x4qh krb5kdc[1319](info): commencing operation
krb5kdc: starting...
W20260809 19:30:17.011139 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:17.117650 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45781
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:45781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:17.193008 1346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:17.193166 1346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:17.193183 1346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:17.194217 1346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:17.194255 1346 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:30:17.194267 1346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:17.194279 1346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:17.194314 1346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:17.194340 1346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:17.195695 1346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:45781
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:17.195924 1346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:17.196142 1346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:17.200680 1355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.200747 1356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.200753 1346 server_base.cc:1034] running on GCE node
W20260809 19:30:17.200681 1359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.201232 1346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:17.201515 1346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:17.202693 1346 hybrid_clock.cc:648] HybridClock initialized: now 1786303817202680 us; error 45 us; skew 500 ppm
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:17.207924 1346 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:17.208359 1346 webserver.cc:460] Webserver started at http://127.31.57.254:45169/ using document root <none> and password file <none>
I20260809 19:30:17.208583 1346 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:17.208647 1346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:17.208782 1346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:17.209688 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "42ee30eb185e4d9f94069faac0238243"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "62e143effb54fa45b9a6b7764888c6a4"
server_key_iv: "b0ec5e386b2f85f8e49a41fec67b5d7d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.210026 1346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "42ee30eb185e4d9f94069faac0238243"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "62e143effb54fa45b9a6b7764888c6a4"
server_key_iv: "b0ec5e386b2f85f8e49a41fec67b5d7d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.211248 1346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:30:17.212025 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.212208 1346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:17.212302 1346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "42ee30eb185e4d9f94069faac0238243"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "62e143effb54fa45b9a6b7764888c6a4"
server_key_iv: "b0ec5e386b2f85f8e49a41fec67b5d7d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.212378 1346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:17.233108 1346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:17.234810 1346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:17.234961 1346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:17.239770 1346 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:45781
I20260809 19:30:17.239815 1459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:45781 every 8 connection(s)
I20260809 19:30:17.240103 1346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:17.240659 1461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.241737 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1346
I20260809 19:30:17.241834 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:17.241974 31975 external_mini_cluster.cc:1428] Setting key 48cb69c5d17ed06f938c9d5c62a2ec8e
I20260809 19:30:17.243070 1461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Bootstrap starting.
I20260809 19:30:17.243765 1461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.244102 1461 log.cc:824] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:17.244779 1461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: No bootstrap required, opened a new log
I20260809 19:30:17.247103 1461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } }
I20260809 19:30:17.247273 1461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.247311 1461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42ee30eb185e4d9f94069faac0238243, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.247468 1461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } }
I20260809 19:30:17.247558 1461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:17.247596 1461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:17.247643 1461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:17.248239 1461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } }
I20260809 19:30:17.248327 1461 leader_election.cc:302] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42ee30eb185e4d9f94069faac0238243; no voters:
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:17.248543 1461 leader_election.cc:288] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:17.248569 1470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:17.248728 1470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [term 1 LEADER]: Becoming Leader. State: Replica: 42ee30eb185e4d9f94069faac0238243, State: Running, Role: LEADER
I20260809 19:30:17.248878 1470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } }
I20260809 19:30:17.248881 1461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:17.249328 1472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42ee30eb185e4d9f94069faac0238243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } } }
I20260809 19:30:17.249431 1472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:17.249328 1473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42ee30eb185e4d9f94069faac0238243. Latest consensus state: current_term: 1 leader_uuid: "42ee30eb185e4d9f94069faac0238243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42ee30eb185e4d9f94069faac0238243" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45781 } } }
I20260809 19:30:17.249720 1473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:17.249773 1479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:17.250229 1479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:17.252009 1479 catalog_manager.cc:1347] Generated new cluster ID: 672db992b530466fbfbd902de1d90b7d
I20260809 19:30:17.252053 1479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:17.252789 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.242444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.31.57.254:45781)
0809 19:30:17.242612 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.242620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.242651 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:30:17.243009 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.243014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.243052 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.243199 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.243978 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.244954 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.246038 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.246458 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.246544 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.249090 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.249094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.249095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.249105 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.251219 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.251661 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.251664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.251665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.251772 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.251998 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.251999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.251999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.252245 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.252398 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.252508 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.252640 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260809 19:30:17.270769 1479 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:17.271222 1479 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:17.278414 1479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Generated new TSK 0
I20260809 19:30:17.278575 1479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:17.307441 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:45781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:17.395002 1509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:17.395197 1509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:17.395219 1509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:17.396390 1509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:17.396433 1509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:17.396453 1509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:17.396471 1509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:17.398010 1509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:17.398248 1509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:17.398468 1509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:17.403157 1521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.403167 1518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.403313 1509 server_base.cc:1034] running on GCE node
W20260809 19:30:17.403167 1517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.403616 1509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:17.403858 1509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:17.405015 1509 hybrid_clock.cc:648] HybridClock initialized: now 1786303817405004 us; error 41 us; skew 500 ppm
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:17.407258 1509 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:17.407663 1509 webserver.cc:460] Webserver started at http://127.31.57.193:38507/ using document root <none> and password file <none>
I20260809 19:30:17.407899 1509 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:17.407964 1509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:17.408107 1509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:17.409025 1509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "2d10ff6aefa640db936c280718dc2abe"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "87b4f0f0e9b3cfe6f8458d23b16f23a0"
server_key_iv: "aafa183245512e4f14886723e89ae9fa"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.409358 1509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "2d10ff6aefa640db936c280718dc2abe"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "87b4f0f0e9b3cfe6f8458d23b16f23a0"
server_key_iv: "aafa183245512e4f14886723e89ae9fa"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.410558 1509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:17.411213 1529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.411417 1509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:17.411516 1509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "2d10ff6aefa640db936c280718dc2abe"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "87b4f0f0e9b3cfe6f8458d23b16f23a0"
server_key_iv: "aafa183245512e4f14886723e89ae9fa"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.411598 1509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:17.424047 1509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:17.425603 1509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:17.425762 1509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:17.426229 1509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:17.426293 1509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.426348 1509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:17.426390 1509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.432885 1509 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:46805
I20260809 19:30:17.432960 1700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:46805 every 8 connection(s)
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:17.433284 1509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:17.436244 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.426777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:42439 (local address 127.31.57.254:45781)
0809 19:30:17.426850 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.426853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.427048 (+ 195us) server_negotiation.cc:407] Connection header received
0809 19:30:17.427536 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.427537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.427549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.427616 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.428399 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.429107 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.430606 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.430947 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.431020 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.433564 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.433566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.433566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.433575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.434859 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.435241 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.435242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.435243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.435316 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.435591 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.435592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.435592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.435709 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.435784 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.436054 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.436159 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 19:30:17.436362 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.426738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.193:42439)
0809 19:30:17.426914 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.426950 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.427429 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.427728 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.427731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.428039 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.428290 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.428296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.429241 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.429242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.430514 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.430517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.431050 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.431051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.431190 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.431712 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.431722 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.433471 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.434944 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.434952 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.434954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.435165 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.435404 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.435405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.435405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.435516 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.435784 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.435792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.435919 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.436016 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.436170 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260809 19:30:17.436849 1701 heartbeater.cc:347] Connected to a master server at 127.31.57.254:45781
I20260809 19:30:17.436944 1701 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:17.437114 1701 heartbeater.cc:511] Master 127.31.57.254:45781 requested a full tablet report, sending...
I20260809 19:30:17.437645 1391 ts_manager.cc:194] Registered new tserver with Master: 2d10ff6aefa640db936c280718dc2abe (127.31.57.193:46805)
I20260809 19:30:17.438535 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:42439
I20260809 19:30:17.442446 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1509
I20260809 19:30:17.442545 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:17.442726 31975 external_mini_cluster.cc:1428] Setting key ad9edadac399e5ccd26fa7099b45098a
I20260809 19:30:17.442972 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.437787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.193:48549)
0809 19:30:17.437937 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.437947 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.438010 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.438124 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.438124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.438316 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.438375 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.438376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.439137 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.439138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.439716 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.439718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.440018 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.440018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.440076 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.440404 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.440407 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.440722 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.441966 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.441966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.441967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.442122 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.442308 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.442309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.442309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.442359 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.442638 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.442639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.442693 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.442750 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.442873 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260809 19:30:17.442970 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.437837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:48549 (local address 127.31.57.254:45781)
0809 19:30:17.437876 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.437878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.438013 (+ 135us) server_negotiation.cc:407] Connection header received
0809 19:30:17.438064 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.438065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.438073 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.438117 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.438433 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.439036 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.439783 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.439953 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.440004 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.440793 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.440794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.440794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.440799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.441897 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.442181 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.442182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.442182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.442228 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.442421 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.442421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.442422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.442553 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.442622 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.442799 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.442886 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:17.467042 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:45781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:17.547147 1714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:17.547314 1714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:17.547335 1714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:17.548377 1714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:17.548414 1714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:17.548431 1714 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:17.548471 1714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:17.549822 1714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:17.550033 1714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:17.550243 1714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:30:17.554992 1714 server_base.cc:1034] running on GCE node
W20260809 19:30:17.554949 1724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.554961 1721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.554948 1722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.555438 1714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:17.555621 1714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:17.556757 1714 hybrid_clock.cc:648] HybridClock initialized: now 1786303817556745 us; error 27 us; skew 500 ppm
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:17.558911 1714 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:17.559242 1714 webserver.cc:460] Webserver started at http://127.31.57.194:45481/ using document root <none> and password file <none>
I20260809 19:30:17.559461 1714 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:17.559504 1714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:17.559592 1714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:17.560526 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "9d08a03eb4d34479b89447330892b364"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "293b39f5b7de4676907726b45073a67d"
server_key_iv: "9a25ac40347df10329893e90c9fb9a0d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.560823 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "9d08a03eb4d34479b89447330892b364"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "293b39f5b7de4676907726b45073a67d"
server_key_iv: "9a25ac40347df10329893e90c9fb9a0d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.562146 1714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:17.562784 1735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.562937 1714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.563014 1714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "9d08a03eb4d34479b89447330892b364"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "293b39f5b7de4676907726b45073a67d"
server_key_iv: "9a25ac40347df10329893e90c9fb9a0d"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.563063 1714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:17.573038 1714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:17.574573 1714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:17.574671 1714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:17.575078 1714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:17.575130 1714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.575151 1714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:17.575188 1714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:17.581362 1714 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:41887
I20260809 19:30:17.581419 1927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:41887 every 8 connection(s)
I20260809 19:30:17.581750 1714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:17.584369 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.575585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:43813 (local address 127.31.57.254:45781)
0809 19:30:17.575627 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.575629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.575856 (+ 227us) server_negotiation.cc:407] Connection header received
0809 19:30:17.576293 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.576295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.576307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.576361 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.577069 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.577805 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.579198 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.579545 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.579623 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.581846 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.581848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.581848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.581856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.583167 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.583611 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.583611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.583612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.583660 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.583867 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.583867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.583867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.583957 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.584002 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.584234 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.584296 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:17.584597 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.575584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.194:43813)
0809 19:30:17.575745 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.575775 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.576193 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.576421 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.576425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.576740 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.576983 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.576989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.577898 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.577900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.579100 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.579103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.579630 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.579631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.579727 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.580234 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.580244 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.581749 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.583236 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.583242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.583244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.583547 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.583719 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.583720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.583720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.583818 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.584009 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.584013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.584145 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.584221 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.584334 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260809 19:30:17.585073 1928 heartbeater.cc:347] Connected to a master server at 127.31.57.254:45781
I20260809 19:30:17.585170 1928 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:17.585351 1928 heartbeater.cc:511] Master 127.31.57.254:45781 requested a full tablet report, sending...
I20260809 19:30:17.585757 1391 ts_manager.cc:194] Registered new tserver with Master: 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887)
I20260809 19:30:17.586233 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:43813
I20260809 19:30:17.590799 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.586108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:58543 (local address 127.31.57.254:45781)
0809 19:30:17.586145 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.586146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.586237 (+ 91us) server_negotiation.cc:407] Connection header received
0809 19:30:17.586297 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.586298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.586307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.586356 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.586747 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.587167 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.587886 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.588056 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.588109 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.588895 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.588896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.588896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.588901 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.589952 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.590228 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.590228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.590229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.590266 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.590389 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.590389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.590389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.590514 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.590577 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.590696 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.590747 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:17.590818 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.586046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.194:58543)
0809 19:30:17.586168 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.586178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.586234 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.586397 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.586399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.586600 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.586664 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.586667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.587257 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.587258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.587818 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.587820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.588122 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.588123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.588182 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.588524 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.588528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.588825 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.590016 (+ 1191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.590017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.590017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.590173 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.590313 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.590314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.590314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.590357 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.590586 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.590587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.590632 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.590681 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.590751 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260809 19:30:17.591902 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1714
I20260809 19:30:17.591995 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:17.592193 31975 external_mini_cluster.cc:1428] Setting key 031113df9df46c5cba5d0c9e7a598c57
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:17.616123 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:45781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:17.692226 1943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:17.692405 1943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:17.692428 1943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:17.693475 1943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:17.693506 1943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:17.693523 1943 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:17.693539 1943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:17.694734 1943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:17.694895 1943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:17.695052 1943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:17.699658 1952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.699671 1954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:17.699672 1950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:17.699942 1943 server_base.cc:1034] running on GCE node
I20260809 19:30:17.700136 1943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:17.700390 1943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:17.701538 1943 hybrid_clock.cc:648] HybridClock initialized: now 1786303817701523 us; error 26 us; skew 500 ppm
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:17.703913 1943 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:17.704320 1943 webserver.cc:460] Webserver started at http://127.31.57.195:35333/ using document root <none> and password file <none>
I20260809 19:30:17.704550 1943 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:17.704617 1943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:17.704746 1943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:17.705739 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "2a9243aaed414226b0505c963e58d818"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "b8494bae4c3f948abb526f804cf698ae"
server_key_iv: "d66c2b7af9c2a030796c854868f89ad4"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.706102 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "2a9243aaed414226b0505c963e58d818"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "b8494bae4c3f948abb526f804cf698ae"
server_key_iv: "d66c2b7af9c2a030796c854868f89ad4"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.707322 1943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:17.708009 1963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.708173 1943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:17.708264 1943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "2a9243aaed414226b0505c963e58d818"
format_stamp: "Formatted at 2026-08-09 19:30:17 on dist-test-slave-x4qh"
server_key: "b8494bae4c3f948abb526f804cf698ae"
server_key_iv: "d66c2b7af9c2a030796c854868f89ad4"
server_key_version: "encryptionkey@0"
I20260809 19:30:17.708344 1943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:17.725471 1943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:17.727087 1943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:17.727233 1943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:17.727784 1943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:17.727847 1943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.727886 1943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:17.727912 1943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:17.734697 1943 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:45825
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:17.734763 2140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:45825 every 8 connection(s)
I20260809 19:30:17.735026 1943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:17.738021 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.728295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:47607 (local address 127.31.57.254:45781)
0809 19:30:17.728365 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.728367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.728578 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:30:17.729200 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.729202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.729212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.729271 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.730091 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.730839 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.732481 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.732741 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.732809 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.735202 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.735204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.735204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.735211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.736621 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.737019 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.737021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.737021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.737078 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.737349 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.737350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.737351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.737479 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.737573 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.737850 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.737934 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:30:17.738178 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.728243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.195:47607)
0809 19:30:17.728442 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.728485 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.729094 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.729354 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.729359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.729715 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.729991 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.729997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.730939 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.730940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.732373 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.732376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.732805 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.732806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.732950 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.733433 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.733445 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.735128 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.736720 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.736727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.736729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.736932 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.737148 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.737149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.737149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.737250 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.737578 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.737584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.737713 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.737823 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.737965 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260809 19:30:17.738708 2141 heartbeater.cc:347] Connected to a master server at 127.31.57.254:45781
I20260809 19:30:17.738802 2141 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:17.738960 2141 heartbeater.cc:511] Master 127.31.57.254:45781 requested a full tablet report, sending...
I20260809 19:30:17.739316 1392 ts_manager.cc:194] Registered new tserver with Master: 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825)
I20260809 19:30:17.739830 1392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:47607
I20260809 19:30:17.740374 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1943
I20260809 19:30:17.740473 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:17.740667 31975 external_mini_cluster.cc:1428] Setting key 926361846615bea0917845aa66dcb284
I20260809 19:30:17.741411 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:30:17.745637 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.739643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.31.57.195:50609)
0809 19:30:17.739994 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.740004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.740053 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.740194 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.740195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.740384 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:17.740474 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.740476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.741406 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.741407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.741938 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.741939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.742383 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.742384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.742438 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.742844 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:17.742848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:17.743113 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:17.744656 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.744657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.744658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.744891 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.745089 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:17.745089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:17.745089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:17.745136 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:17.745402 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:17.745403 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:17.745446 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.745504 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.745579 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260809 19:30:17.745744 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.739693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50609 (local address 127.31.57.254:45781)
0809 19:30:17.739723 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.739724 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.740061 (+ 337us) server_negotiation.cc:407] Connection header received
0809 19:30:17.740102 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.740103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.740113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.740176 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.740579 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.741307 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.742022 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.742304 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.742379 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.743207 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.743208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.743208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.743215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.744544 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.744966 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.744967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.744968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.745027 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.745198 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.745199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.745199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.745342 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.745415 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.745553 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.745627 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:30:17 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:17.764125 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.757960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.31.57.254:45781)
0809 19:30:17.758018 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.758019 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.758026 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:17.758043 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.758044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.758051 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.758098 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:17.758595 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.759015 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.759721 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.759878 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.759925 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.761845 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:17.761847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:17.761847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:17.761853 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:17.763252 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.763594 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.763595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.763595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.763643 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:17.763789 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:17.763789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:17.763789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:17.763870 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:17.763936 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.764025 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.764077 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:30:17.765467 1392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:17.766973 1392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:17.772744 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.768896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.31.57.195:45825)
0809 19:30:17.769354 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.769358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.769364 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:30:17.769391 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.769392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.769402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.769484 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:17.769653 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770316 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.771599 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772322 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772408 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772462 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.772623 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.772659 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":76,"threads_started":1}
I20260809 19:30:17.772737 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.768805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:45825 (local address 127.0.0.1:42844)
0809 19:30:17.768970 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.769004 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.769064 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.769514 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.769517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.769519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:17.769576 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.769578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770406 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.770407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.771505 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.771508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772410 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.772494 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772506 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.772606 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.772659 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260809 19:30:17.772897 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.768806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:46805 (local address 127.0.0.1:41512)
0809 19:30:17.769129 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.769140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.769196 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.769641 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.769641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.769643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:17.769672 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.769673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770833 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.770834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.771728 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.771731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772646 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.772722 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772726 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.772774 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.772824 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":26,"spinlock_wait_cycles":384,"thread_start_us":84,"threads_started":1}
I20260809 19:30:17.773072 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.769012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:41887 (local address 127.0.0.1:49870)
0809 19:30:17.769198 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.769207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.769247 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.769593 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.769594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.769596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:17.769647 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.769649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770828 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.770829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.771903 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.771906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772863 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.772920 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772924 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.772975 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.773014 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":25,"thread_start_us":48,"threads_started":1}
I20260809 19:30:17.773265 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.769011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.31.57.193:46805)
0809 19:30:17.769416 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.769420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.769430 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:17.769468 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.769469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.769483 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.769578 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:17.769741 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770748 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.771803 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772489 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772561 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772609 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.773056 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.773100 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":105,"threads_started":1}
I20260809 19:30:17.773339 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.769202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.31.57.194:41887)
0809 19:30:17.769361 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.769366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.769377 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:30:17.769417 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.769420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.769436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.769527 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:17.769728 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.770731 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.771989 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.772721 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.772782 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.772831 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.773178 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.773222 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260809 19:30:17.773540 2042 tablet_service.cc:1467] Processing CreateTablet for tablet c778dda637394f37a35a51176104ffeb (DEFAULT_TABLE table=test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.773890 2042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c778dda637394f37a35a51176104ffeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.773828 1604 tablet_service.cc:1467] Processing CreateTablet for tablet c778dda637394f37a35a51176104ffeb (DEFAULT_TABLE table=test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.773891 1817 tablet_service.cc:1467] Processing CreateTablet for tablet c778dda637394f37a35a51176104ffeb (DEFAULT_TABLE table=test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.774132 1604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c778dda637394f37a35a51176104ffeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.774195 1817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c778dda637394f37a35a51176104ffeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.776732 2175 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Bootstrap starting.
I20260809 19:30:17.777338 2175 tablet_bootstrap.cc:654] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.777444 2177 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Bootstrap starting.
I20260809 19:30:17.777628 2176 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Bootstrap starting.
I20260809 19:30:17.777675 2175 log.cc:824] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:17.778110 2177 tablet_bootstrap.cc:654] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.778381 2175 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: No bootstrap required, opened a new log
I20260809 19:30:17.778460 2175 ts_tablet_manager.cc:1397] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:17.778445 2177 log.cc:824] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:17.778501 2176 tablet_bootstrap.cc:654] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.778847 2176 log.cc:824] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:17.779217 2177 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: No bootstrap required, opened a new log
I20260809 19:30:17.779325 2177 ts_tablet_manager.cc:1397] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:17.779553 2176 tablet_bootstrap.cc:492] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: No bootstrap required, opened a new log
I20260809 19:30:17.779606 2176 ts_tablet_manager.cc:1397] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:17.780547 2175 raft_consensus.cc:348] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.780702 2175 raft_consensus.cc:374] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.780737 2175 raft_consensus.cc:729] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.780848 2175 consensus_queue.cc:260] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.780907 2176 raft_consensus.cc:348] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.781059 2176 raft_consensus.cc:374] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.781092 2175 ts_tablet_manager.cc:1428] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:30:17.781102 2176 raft_consensus.cc:729] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9243aaed414226b0505c963e58d818, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.781206 1701 heartbeater.cc:503] Master 127.31.57.254:45781 was elected leader, sending a full tablet report...
I20260809 19:30:17.781225 2176 consensus_queue.cc:260] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.781510 2176 ts_tablet_manager.cc:1428] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:17.781563 2141 heartbeater.cc:503] Master 127.31.57.254:45781 was elected leader, sending a full tablet report...
I20260809 19:30:17.782023 2177 raft_consensus.cc:348] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.782181 2177 raft_consensus.cc:374] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.782224 2177 raft_consensus.cc:729] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d08a03eb4d34479b89447330892b364, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.782335 2177 consensus_queue.cc:260] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.782598 2177 ts_tablet_manager.cc:1428] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:30:17.782673 1928 heartbeater.cc:503] Master 127.31.57.254:45781 was elected leader, sending a full tablet report...
W20260809 19:30:17.832362 1930 tablet.cc:2369] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:17.909313 2183 raft_consensus.cc:482] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:17.909425 2183 raft_consensus.cc:504] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.909734 2183 leader_election.cc:288] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825), 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887)
I20260809 19:30:17.913551 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.910019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:40083 (local address 127.31.57.195:45825)
0809 19:30:17.910076 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.910079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.910191 (+ 112us) server_negotiation.cc:407] Connection header received
0809 19:30:17.910248 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.910249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.910262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.910349 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:17.910549 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.911197 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.912687 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.913217 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.913299 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.913306 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.913453 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.913490 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:30:17.913551 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.910000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:32771 (local address 127.31.57.194:41887)
0809 19:30:17.910042 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.910044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.910053 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:17.910151 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.910153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.910162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.910237 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:17.910454 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.911218 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.912687 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.913217 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.913290 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.913299 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.913453 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.913490 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:30:17.913565 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.909927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:41887 (local address 127.31.57.193:32771)
0809 19:30:17.910004 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.910015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.910063 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.910259 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.910260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.910263 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:17.910363 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.910365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.911294 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.911294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.912559 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.912562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.913299 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.913300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.913372 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.913376 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.913431 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.913480 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":15}
I20260809 19:30:17.913578 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.909948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:45825 (local address 127.31.57.193:40083)
0809 19:30:17.910115 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:17.910123 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:17.910196 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:17.910355 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:17.910356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:17.910370 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:17.910442 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.910443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.911290 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.911291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.912559 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:17.912562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.913325 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.913325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:17.913386 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.913389 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:17.913434 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:30:17.913480 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":81,"thread_start_us":121,"threads_started":1}
I20260809 19:30:17.913910 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c778dda637394f37a35a51176104ffeb" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364" is_pre_election: true
I20260809 19:30:17.914084 1847 raft_consensus.cc:2454] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:17.914333 1530 leader_election.cc:302] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d10ff6aefa640db936c280718dc2abe, 9d08a03eb4d34479b89447330892b364; no voters:
I20260809 19:30:17.914479 2183 raft_consensus.cc:2781] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:17.914455 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c778dda637394f37a35a51176104ffeb" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818" is_pre_election: true
I20260809 19:30:17.914537 2183 raft_consensus.cc:482] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:17.914567 2183 raft_consensus.cc:3037] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:17.914587 2081 raft_consensus.cc:2454] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:17.915561 2183 raft_consensus.cc:504] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.915706 2183 leader_election.cc:288] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Requested vote from peers 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825), 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887)
I20260809 19:30:17.915882 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c778dda637394f37a35a51176104ffeb" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818"
I20260809 19:30:17.915961 2081 raft_consensus.cc:3037] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:17.916637 2081 raft_consensus.cc:2454] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:17.916750 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c778dda637394f37a35a51176104ffeb" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364"
I20260809 19:30:17.916847 1847 raft_consensus.cc:3037] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:17.917168 1531 leader_election.cc:302] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9243aaed414226b0505c963e58d818, 2d10ff6aefa640db936c280718dc2abe; no voters:
I20260809 19:30:17.917273 2183 raft_consensus.cc:2781] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:17.917392 2183 raft_consensus.cc:686] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 1 LEADER]: Becoming Leader. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Running, Role: LEADER
I20260809 19:30:17.917613 2183 consensus_queue.cc:237] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:17.917737 1847 raft_consensus.cc:2454] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:17.918552 1391 catalog_manager.cc:5495] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d10ff6aefa640db936c280718dc2abe (127.31.57.193). New cstate: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } health_report { overall_health: UNKNOWN } } }
W20260809 19:30:17.934003 1703 tablet.cc:2369] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:17.958679 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:17.953812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41516 (local address 127.31.57.193:46805)
0809 19:30:17.954044 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:17.954048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:17.954070 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:30:17.954101 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:17.954102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:17.954118 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:17.954199 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:17.954391 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.955203 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.957665 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:17.957856 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:17.957926 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:17.958233 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:17.958374 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:17.958462 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:17.958559 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:30:17.958587 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260809 19:30:17.960372 2081 raft_consensus.cc:1261] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Refusing update from remote peer 2d10ff6aefa640db936c280718dc2abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:17.960673 1847 raft_consensus.cc:1261] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Refusing update from remote peer 2d10ff6aefa640db936c280718dc2abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:17.960836 2183 consensus_queue.cc:1035] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:17.960951 2190 consensus_queue.cc:1035] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:17.963341 2196 mvcc.cc:205] Tried to move back new op lower bound from 7316700438363414528 to 7316700438191087616. Current Snapshot: MvccSnapshot[applied={T|T < 7316700438363414528}]
I20260809 19:30:17.964213 2198 mvcc.cc:205] Tried to move back new op lower bound from 7316700438363414528 to 7316700438191087616. Current Snapshot: MvccSnapshot[applied={T|T < 7316700438363414528}]
I20260809 19:30:17.965217 2197 mvcc.cc:205] Tried to move back new op lower bound from 7316700438363414528 to 7316700438191087616. Current Snapshot: MvccSnapshot[applied={T|T < 7316700438363414528 or (T in {7316700438363414528})}]
I20260809 19:30:17.966750 1392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:17.966845 1392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:17.967613 1392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Sending DeleteTablet for 3 replicas of tablet c778dda637394f37a35a51176104ffeb
I20260809 19:30:17.968039 2042 tablet_service.cc:1514] Processing DeleteTablet for tablet c778dda637394f37a35a51176104ffeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:17 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42844
I20260809 19:30:17.968025 1604 tablet_service.cc:1514] Processing DeleteTablet for tablet c778dda637394f37a35a51176104ffeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:17 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:41512
I20260809 19:30:17.968240 2211 tablet_replica.cc:330] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: stopping tablet replica
I20260809 19:30:17.968278 1817 tablet_service.cc:1514] Processing DeleteTablet for tablet c778dda637394f37a35a51176104ffeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:17 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:49870
I20260809 19:30:17.968345 2211 raft_consensus.cc:2229] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:17.968451 2211 raft_consensus.cc:2258] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:17.968659 1392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:17.968804 1392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:17.969177 2211 ts_tablet_manager.cc:1905] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:17.970038 1604 tablet_service.cc:1467] Processing CreateTablet for tablet f0c9d78716b248cca76f56ac218d2447 (DEFAULT_TABLE table=test-table [id=8f4c51cbcd81450ca40255408aee12d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.970170 1604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c9d78716b248cca76f56ac218d2447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.971544 2175 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Bootstrap starting.
I20260809 19:30:17.972036 2175 tablet_bootstrap.cc:654] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.972379 2211 ts_tablet_manager.cc:1918] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:17.972509 2211 log.cc:1196] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/c778dda637394f37a35a51176104ffeb
I20260809 19:30:17.972769 2175 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: No bootstrap required, opened a new log
I20260809 19:30:17.972826 2175 ts_tablet_manager.cc:1397] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:17.972976 2211 ts_tablet_manager.cc:1939] T c778dda637394f37a35a51176104ffeb P 2a9243aaed414226b0505c963e58d818: Deleting consensus metadata
I20260809 19:30:17.972985 2175 raft_consensus.cc:348] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.973008 1817 tablet_service.cc:1467] Processing CreateTablet for tablet f0c9d78716b248cca76f56ac218d2447 (DEFAULT_TABLE table=test-table [id=8f4c51cbcd81450ca40255408aee12d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.973063 2175 raft_consensus.cc:374] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.973093 2175 raft_consensus.cc:729] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.973136 1817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c9d78716b248cca76f56ac218d2447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.973131 2175 consensus_queue.cc:260] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.973217 2175 ts_tablet_manager.cc:1428] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:17.974503 2177 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Bootstrap starting.
I20260809 19:30:17.974664 2042 tablet_service.cc:1467] Processing CreateTablet for tablet f0c9d78716b248cca76f56ac218d2447 (DEFAULT_TABLE table=test-table [id=8f4c51cbcd81450ca40255408aee12d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:17.974787 2042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0c9d78716b248cca76f56ac218d2447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:17.974917 2177 tablet_bootstrap.cc:654] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.975560 2177 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: No bootstrap required, opened a new log
I20260809 19:30:17.975625 2177 ts_tablet_manager.cc:1397] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:17.975775 2177 raft_consensus.cc:348] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.975847 2177 raft_consensus.cc:374] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.975875 2177 raft_consensus.cc:729] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d08a03eb4d34479b89447330892b364, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.975920 2177 consensus_queue.cc:260] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.976008 2177 ts_tablet_manager.cc:1428] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:17.975984 2176 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Bootstrap starting.
I20260809 19:30:17.976415 2176 tablet_bootstrap.cc:654] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:17.976835 1369 catalog_manager.cc:4841] TS 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825): tablet c778dda637394f37a35a51176104ffeb (table test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]) successfully deleted
I20260809 19:30:17.977723 2176 tablet_bootstrap.cc:492] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: No bootstrap required, opened a new log
I20260809 19:30:17.977788 2176 ts_tablet_manager.cc:1397] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:17.977926 2176 raft_consensus.cc:348] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.977990 2176 raft_consensus.cc:374] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:17.978008 2176 raft_consensus.cc:729] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9243aaed414226b0505c963e58d818, State: Initialized, Role: FOLLOWER
I20260809 19:30:17.978044 2176 consensus_queue.cc:260] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:17.978130 2176 ts_tablet_manager.cc:1428] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:17.978356 2213 tablet_replica.cc:330] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: stopping tablet replica
I20260809 19:30:17.978596 2212 tablet_replica.cc:330] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: stopping tablet replica
I20260809 19:30:17.978736 2212 raft_consensus.cc:2229] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:17.978847 2212 raft_consensus.cc:2258] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:17.978991 2213 raft_consensus.cc:2229] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:17.979096 2213 raft_consensus.cc:2258] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:17.979347 2212 ts_tablet_manager.cc:1905] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:17.979940 2213 ts_tablet_manager.cc:1905] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:17.981046 2212 ts_tablet_manager.cc:1918] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:17.981128 2212 log.cc:1196] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/c778dda637394f37a35a51176104ffeb
I20260809 19:30:17.981395 2213 ts_tablet_manager.cc:1918] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:17.981411 2212 ts_tablet_manager.cc:1939] T c778dda637394f37a35a51176104ffeb P 2d10ff6aefa640db936c280718dc2abe: Deleting consensus metadata
I20260809 19:30:17.981468 2213 log.cc:1196] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/c778dda637394f37a35a51176104ffeb
I20260809 19:30:17.981820 1368 catalog_manager.cc:4841] TS 2d10ff6aefa640db936c280718dc2abe (127.31.57.193:46805): tablet c778dda637394f37a35a51176104ffeb (table test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]) successfully deleted
I20260809 19:30:17.982123 2213 ts_tablet_manager.cc:1939] T c778dda637394f37a35a51176104ffeb P 9d08a03eb4d34479b89447330892b364: Deleting consensus metadata
I20260809 19:30:17.982546 1368 catalog_manager.cc:4841] TS 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887): tablet c778dda637394f37a35a51176104ffeb (table test-table [id=b4a1e8a9e51c473888fa1fb2ec270b36]) successfully deleted
W20260809 19:30:17.985849 2142 tablet.cc:2369] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:18.151895 2190 raft_consensus.cc:482] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:18.151988 2190 raft_consensus.cc:504] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:18.152133 2190 leader_election.cc:288] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887), 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825)
I20260809 19:30:18.152339 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c9d78716b248cca76f56ac218d2447" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818" is_pre_election: true
I20260809 19:30:18.152351 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c9d78716b248cca76f56ac218d2447" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364" is_pre_election: true
I20260809 19:30:18.152426 1847 raft_consensus.cc:2454] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:18.152463 2081 raft_consensus.cc:2454] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:18.152607 1530 leader_election.cc:302] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d10ff6aefa640db936c280718dc2abe, 9d08a03eb4d34479b89447330892b364; no voters:
I20260809 19:30:18.152732 2190 raft_consensus.cc:2781] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:18.152792 2190 raft_consensus.cc:482] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:18.152807 2190 raft_consensus.cc:3037] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.153512 2190 raft_consensus.cc:504] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:18.153649 2190 leader_election.cc:288] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Requested vote from peers 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887), 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825)
I20260809 19:30:18.153779 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c9d78716b248cca76f56ac218d2447" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818"
I20260809 19:30:18.153803 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0c9d78716b248cca76f56ac218d2447" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364"
I20260809 19:30:18.153852 1847 raft_consensus.cc:3037] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.153870 2081 raft_consensus.cc:3037] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.154492 2081 raft_consensus.cc:2454] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:18.154506 1847 raft_consensus.cc:2454] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:18.154685 1531 leader_election.cc:302] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9243aaed414226b0505c963e58d818, 2d10ff6aefa640db936c280718dc2abe; no voters:
I20260809 19:30:18.154780 2190 raft_consensus.cc:2781] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:18.154842 2190 raft_consensus.cc:686] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 1 LEADER]: Becoming Leader. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Running, Role: LEADER
I20260809 19:30:18.154892 2190 consensus_queue.cc:237] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } }
I20260809 19:30:18.155535 1392 catalog_manager.cc:5495] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d10ff6aefa640db936c280718dc2abe (127.31.57.193). New cstate: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } health_report { overall_health: UNKNOWN } } }
Aug 09 19:30:18 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:18.186848 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.180077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.31.57.254:45781)
0809 19:30:18.180167 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:18.180170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:18.180265 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:30:18.180347 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:18.180348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:18.180357 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:18.180428 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:18.180896 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.181381 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.182076 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.182253 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.182344 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.184304 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:18.184305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:18.184306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:18.184312 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:18.185679 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.186056 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.186057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.186057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.186110 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.186322 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.186323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.186323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.186422 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:18.186494 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:18.186635 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 19:30:18.186715 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 19:30:18.186947 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.179994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45781 (local address 127.0.0.1:53370)
0809 19:30:18.180191 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:18.180201 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:18.180281 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:18.180432 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:18.180433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:18.180757 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:18.180809 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.180811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.181475 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.181476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.181993 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.181995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.182344 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.182345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.182405 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.182889 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:18.182893 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:18.184205 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:18.185767 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:18.185775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:18.185777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:18.185972 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:18.186174 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:18.186175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:18.186176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:18.186236 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:18.186485 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:18.186492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:18.186534 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:30:18.186615 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:30:18.186802 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":75,"thread_start_us":109,"threads_started":1}
I20260809 19:30:18.189041 1392 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53370:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:30:18.189254 1392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:18.190757 1604 tablet_service.cc:1467] Processing CreateTablet for tablet bc458cf834a148b7b72a5b70312560c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=98878dcf08004c4e8b975e10c7b399aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:18.190760 2042 tablet_service.cc:1467] Processing CreateTablet for tablet bc458cf834a148b7b72a5b70312560c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=98878dcf08004c4e8b975e10c7b399aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:18.190800 1817 tablet_service.cc:1467] Processing CreateTablet for tablet bc458cf834a148b7b72a5b70312560c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=98878dcf08004c4e8b975e10c7b399aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:18.190935 1604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc458cf834a148b7b72a5b70312560c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:18.190939 2042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc458cf834a148b7b72a5b70312560c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:18.190948 1817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc458cf834a148b7b72a5b70312560c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:18.192183 2175 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: Bootstrap starting.
I20260809 19:30:18.192528 2175 tablet_bootstrap.cc:654] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:18.192529 2176 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: Bootstrap starting.
I20260809 19:30:18.192984 2176 tablet_bootstrap.cc:654] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:18.193188 2175 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: No bootstrap required, opened a new log
I20260809 19:30:18.193233 2175 ts_tablet_manager.cc:1397] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:18.193346 2177 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: Bootstrap starting.
I20260809 19:30:18.193333 2175 raft_consensus.cc:348] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.193389 2175 raft_consensus.cc:374] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:18.193406 2175 raft_consensus.cc:729] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Initialized, Role: FOLLOWER
I20260809 19:30:18.193454 2175 consensus_queue.cc:260] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.193552 2175 ts_tablet_manager.cc:1428] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:18.193573 2190 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.193655 2190 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.193746 2177 tablet_bootstrap.cc:654] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:18.194306 2176 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: No bootstrap required, opened a new log
I20260809 19:30:18.194355 2176 ts_tablet_manager.cc:1397] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:18.194522 2176 raft_consensus.cc:348] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.194568 2176 raft_consensus.cc:374] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:18.194581 2176 raft_consensus.cc:729] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9243aaed414226b0505c963e58d818, State: Initialized, Role: FOLLOWER
I20260809 19:30:18.194634 2176 consensus_queue.cc:260] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.194725 2176 ts_tablet_manager.cc:1428] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:18.194780 2184 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.194868 2184 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.195192 2177 tablet_bootstrap.cc:492] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: No bootstrap required, opened a new log
I20260809 19:30:18.195242 2177 ts_tablet_manager.cc:1397] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:18.195405 2177 raft_consensus.cc:348] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.195475 2177 raft_consensus.cc:374] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:18.195508 2177 raft_consensus.cc:729] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d08a03eb4d34479b89447330892b364, State: Initialized, Role: FOLLOWER
I20260809 19:30:18.195544 2177 consensus_queue.cc:260] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.195616 2177 ts_tablet_manager.cc:1428] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:18.195681 2186 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.195801 2186 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.562631 2190 raft_consensus.cc:482] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:18.562764 2190 raft_consensus.cc:504] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.562980 2190 leader_election.cc:288] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825), 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887)
I20260809 19:30:18.563160 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc458cf834a148b7b72a5b70312560c8" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818" is_pre_election: true
I20260809 19:30:18.563185 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc458cf834a148b7b72a5b70312560c8" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364" is_pre_election: true
I20260809 19:30:18.563263 2081 raft_consensus.cc:2454] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:18.563263 1847 raft_consensus.cc:2454] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 0.
I20260809 19:30:18.563441 1531 leader_election.cc:302] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9243aaed414226b0505c963e58d818, 2d10ff6aefa640db936c280718dc2abe; no voters:
I20260809 19:30:18.563536 2190 raft_consensus.cc:2781] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:18.563576 2190 raft_consensus.cc:482] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:18.563603 2190 raft_consensus.cc:3037] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.564652 2190 raft_consensus.cc:504] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.564792 2190 leader_election.cc:288] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Requested vote from peers 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825), 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887)
I20260809 19:30:18.564968 2081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc458cf834a148b7b72a5b70312560c8" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a9243aaed414226b0505c963e58d818"
I20260809 19:30:18.565034 2081 raft_consensus.cc:3037] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.565774 2081 raft_consensus.cc:2454] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:18.565975 1531 leader_election.cc:302] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a9243aaed414226b0505c963e58d818, 2d10ff6aefa640db936c280718dc2abe; no voters:
I20260809 19:30:18.566064 2190 raft_consensus.cc:2781] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:18.566121 2190 raft_consensus.cc:686] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [term 1 LEADER]: Becoming Leader. State: Replica: 2d10ff6aefa640db936c280718dc2abe, State: Running, Role: LEADER
I20260809 19:30:18.566152 1847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc458cf834a148b7b72a5b70312560c8" candidate_uuid: "2d10ff6aefa640db936c280718dc2abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d08a03eb4d34479b89447330892b364"
I20260809 19:30:18.566192 2190 consensus_queue.cc:237] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } }
I20260809 19:30:18.566263 1847 raft_consensus.cc:3037] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:18.567229 1847 raft_consensus.cc:2454] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d10ff6aefa640db936c280718dc2abe in term 1.
I20260809 19:30:18.567454 2190 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: TxnStatusTablet state changed. Reason: New leader 2d10ff6aefa640db936c280718dc2abe. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.567533 2190 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:18.567665 2237 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:30:18.567880 2237 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:30:18.567903 1391 catalog_manager.cc:5495] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d10ff6aefa640db936c280718dc2abe (127.31.57.193). New cstate: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:18.578585 2183 consensus_queue.cc:1035] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:18.580056 2183 consensus_queue.cc:1035] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:18.634995 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.632412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.31.57.193:46805)
0809 19:30:18.632602 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:18.632604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:18.632613 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:18.632647 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:18.632647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:18.632657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:18.632741 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:18.632919 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.633405 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.634197 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.634405 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.634486 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.634692 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:18.634731 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:18.634810 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:18.634882 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 19:30:18.634906 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260809 19:30:18.634999 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.632322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:46805 (local address 127.0.0.1:41526)
0809 19:30:18.632407 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:18.632421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:18.632470 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:18.632756 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:18.632757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:18.632759 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:18.632824 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.632828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.633494 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.633495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.634085 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.634087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.634500 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.634501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.634572 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.634588 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:18.634813 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:18.634824 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 19:30:18.634878 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:30:18.634928 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:30:18.636211 1847 raft_consensus.cc:1261] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Refusing update from remote peer 2d10ff6aefa640db936c280718dc2abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:18.636219 2081 raft_consensus.cc:1261] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Refusing update from remote peer 2d10ff6aefa640db936c280718dc2abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:18.636438 2183 consensus_queue.cc:1035] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:18.636545 2190 consensus_queue.cc:1035] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:18.637153 2186 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: TxnStatusTablet state changed. Reason: New leader 2d10ff6aefa640db936c280718dc2abe. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.637180 2184 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: TxnStatusTablet state changed. Reason: New leader 2d10ff6aefa640db936c280718dc2abe. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.637253 2186 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.637264 2184 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.638126 2183 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.638217 2183 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:18.638211 2190 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.638283 2190 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2d10ff6aefa640db936c280718dc2abe: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:18.638226 2184 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.638309 2184 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 2a9243aaed414226b0505c963e58d818: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.638996 2186 tablet_replica.cc:439] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d10ff6aefa640db936c280718dc2abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d10ff6aefa640db936c280718dc2abe" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 46805 } } peers { permanent_uuid: "2a9243aaed414226b0505c963e58d818" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45825 } } peers { permanent_uuid: "9d08a03eb4d34479b89447330892b364" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41887 } } }
I20260809 19:30:18.639072 2186 tablet_replica.cc:442] T bc458cf834a148b7b72a5b70312560c8 P 9d08a03eb4d34479b89447330892b364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:18.645153 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.642774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:40615 (local address 127.31.57.193:46805)
0809 19:30:18.642814 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:18.642816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:18.642946 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:30:18.643024 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:18.643025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:18.643034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:18.643086 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:18.643185 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.643662 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.644381 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.644576 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.644665 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.644830 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:18.644858 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:18.644913 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:18.644969 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:30:18.645006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":19}
I20260809 19:30:18.645160 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.642655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:46805 (local address 127.31.57.193:40615)
0809 19:30:18.642861 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:18.642878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:18.642971 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:18.643088 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:18.643095 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:18.643096 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:18.643131 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.643133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.643751 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.643751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.644270 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:18.644271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.644666 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.644667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:18.644714 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.644728 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:18.644913 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:18.644914 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:18.644967 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:30:18.645019 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260809 19:30:18.659713 1391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:18.659796 1391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:18.660411 1391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42ee30eb185e4d9f94069faac0238243: Sending DeleteTablet for 3 replicas of tablet f0c9d78716b248cca76f56ac218d2447
I20260809 19:30:18.660686 2042 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c9d78716b248cca76f56ac218d2447 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:18 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42844
I20260809 19:30:18.660745 1817 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c9d78716b248cca76f56ac218d2447 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:18 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:49870
I20260809 19:30:18.660745 1604 tablet_service.cc:1514] Processing DeleteTablet for tablet f0c9d78716b248cca76f56ac218d2447 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:18 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:41512
I20260809 19:30:18.660921 2272 tablet_replica.cc:330] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: stopping tablet replica
I20260809 19:30:18.660974 2272 raft_consensus.cc:2229] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:18.661010 2272 raft_consensus.cc:2258] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:18.661249 2272 ts_tablet_manager.cc:1905] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:18.661536 2273 tablet_replica.cc:330] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: stopping tablet replica
I20260809 19:30:18.661631 2273 raft_consensus.cc:2229] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:18.661760 2273 raft_consensus.cc:2258] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:18.662021 2273 ts_tablet_manager.cc:1905] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:18.663316 2272 ts_tablet_manager.cc:1918] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:18.663425 2272 log.cc:1196] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/f0c9d78716b248cca76f56ac218d2447
I20260809 19:30:18.663661 2272 ts_tablet_manager.cc:1939] T f0c9d78716b248cca76f56ac218d2447 P 9d08a03eb4d34479b89447330892b364: Deleting consensus metadata
I20260809 19:30:18.663734 2273 ts_tablet_manager.cc:1918] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:18.663795 2273 log.cc:1196] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/f0c9d78716b248cca76f56ac218d2447
I20260809 19:30:18.663970 1368 catalog_manager.cc:4841] TS 9d08a03eb4d34479b89447330892b364 (127.31.57.194:41887): tablet f0c9d78716b248cca76f56ac218d2447 (table test-table [id=8f4c51cbcd81450ca40255408aee12d3]) successfully deleted
I20260809 19:30:18.664031 2273 ts_tablet_manager.cc:1939] T f0c9d78716b248cca76f56ac218d2447 P 2d10ff6aefa640db936c280718dc2abe: Deleting consensus metadata
I20260809 19:30:18.664291 2270 tablet_replica.cc:330] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: stopping tablet replica
I20260809 19:30:18.664332 1368 catalog_manager.cc:4841] TS 2d10ff6aefa640db936c280718dc2abe (127.31.57.193:46805): tablet f0c9d78716b248cca76f56ac218d2447 (table test-table [id=8f4c51cbcd81450ca40255408aee12d3]) successfully deleted
I20260809 19:30:18.664377 2270 raft_consensus.cc:2229] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:18.664439 2270 raft_consensus.cc:2258] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:18.664664 2270 ts_tablet_manager.cc:1905] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:18.666268 2270 ts_tablet_manager.cc:1918] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:18.666342 2270 log.cc:1196] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/f0c9d78716b248cca76f56ac218d2447
I20260809 19:30:18.666554 2270 ts_tablet_manager.cc:1939] T f0c9d78716b248cca76f56ac218d2447 P 2a9243aaed414226b0505c963e58d818: Deleting consensus metadata
I20260809 19:30:18.666846 1369 catalog_manager.cc:4841] TS 2a9243aaed414226b0505c963e58d818 (127.31.57.195:45825): tablet f0c9d78716b248cca76f56ac218d2447 (table test-table [id=8f4c51cbcd81450ca40255408aee12d3]) successfully deleted
Aug 09 19:30:18 dist-test-slave-x4qh krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303817, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:18.674856 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.661583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.31.57.193:46805)
0809 19:30:18.661621 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:18.661623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:18.666863 (+ 5240us) server_negotiation.cc:407] Connection header received
0809 19:30:18.666944 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:18.666945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:18.666956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:18.667010 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:18.667623 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.668020 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.668981 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.669140 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.669185 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.671828 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:18.671842 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:18.671842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:18.671863 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:18.673701 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.674070 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.674074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.674074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.674140 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.674332 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.674333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.674333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.674502 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:18.674643 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:18.674695 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 19:30:18.674785 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:30:18.675107 1692 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41536
I20260809 19:30:18.681948 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:18.676033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.31.57.254:45781)
0809 19:30:18.676080 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:18.676082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:18.676105 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:30:18.676161 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:18.676162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:18.676172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:18.676229 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:18.676704 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.677151 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.677849 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:18.678041 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:18.678131 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:18.679307 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:18.679309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:18.679309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:18.679317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:18.680777 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.681146 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.681147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.681148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.681207 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:18.681415 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:18.681416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:18.681416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:18.681529 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:18.681643 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:18.681776 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 19:30:18.681851 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 19:30:18.682255 1391 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53378
I20260809 19:30:18.682775 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1509
I20260809 19:30:18.710481 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1714
I20260809 19:30:18.717252 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1943
I20260809 19:30:18.725606 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1346
2026-08-09T19:30:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2872 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:18 dist-test-slave-x4qh krb5kdc[2282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:18 dist-test-slave-x4qh krb5kdc[2282](info): set up 2 sockets
Aug 09 19:30:18 dist-test-slave-x4qh krb5kdc[2282](info): commencing operation
krb5kdc: starting...
W20260809 19:30:19.885535 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:19 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:19.995625 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43267
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:20.072669 2304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:20.072896 2304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:20.072935 2304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:20.073971 2304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:20.074034 2304 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:30:20.074054 2304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:20.074071 2304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:20.074101 2304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:20.074127 2304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:20.075479 2304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:43267
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:20.075728 2304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:20.075927 2304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:20.080704 2313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.080833 2314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.080914 2304 server_base.cc:1034] running on GCE node
W20260809 19:30:20.080845 2319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.081241 2304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:20.081537 2304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:20.082700 2304 hybrid_clock.cc:648] HybridClock initialized: now 1786303820082678 us; error 36 us; skew 500 ppm
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:20.085035 2304 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:20.085429 2304 webserver.cc:460] Webserver started at http://127.31.57.254:46391/ using document root <none> and password file <none>
I20260809 19:30:20.085656 2304 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:20.085723 2304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:20.085877 2304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:20.086894 2304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "fec4df7e43a64f64b0d94309b3b217a1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "2376239e9e186f3e64c93547e7bd610e"
server_key_iv: "03705b454785e0073810905ef4191a59"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.087245 2304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "fec4df7e43a64f64b0d94309b3b217a1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "2376239e9e186f3e64c93547e7bd610e"
server_key_iv: "03705b454785e0073810905ef4191a59"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.088522 2304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:20.089169 2326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.089345 2304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:20.089442 2304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "fec4df7e43a64f64b0d94309b3b217a1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "2376239e9e186f3e64c93547e7bd610e"
server_key_iv: "03705b454785e0073810905ef4191a59"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.089519 2304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:20.105881 2304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:20.107515 2304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:20.107676 2304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:20.112015 2304 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:43267
I20260809 19:30:20.112125 2423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:43267 every 8 connection(s)
I20260809 19:30:20.112465 2304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:20.113157 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:20.115195 2424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Bootstrap starting.
I20260809 19:30:20.115806 2424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:20.116073 2424 log.cc:824] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:20.116706 2424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: No bootstrap required, opened a new log
I20260809 19:30:20.118701 2424 raft_consensus.cc:348] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } }
I20260809 19:30:20.118824 2424 raft_consensus.cc:374] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:20.118866 2424 raft_consensus.cc:729] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec4df7e43a64f64b0d94309b3b217a1, State: Initialized, Role: FOLLOWER
I20260809 19:30:20.118997 2424 consensus_queue.cc:260] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } }
I20260809 19:30:20.119059 2424 raft_consensus.cc:388] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:20.119082 2424 raft_consensus.cc:482] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:20.119120 2424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:20.119704 2424 raft_consensus.cc:504] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } }
I20260809 19:30:20.119787 2424 leader_election.cc:302] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fec4df7e43a64f64b0d94309b3b217a1; no voters:
I20260809 19:30:20.119941 2424 leader_election.cc:288] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:20.119988 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2304
I20260809 19:30:20.120028 2428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:20.120080 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:20.120177 2424 sys_catalog.cc:564] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:20.120271 31975 external_mini_cluster.cc:1428] Setting key 095c09b4b43245144ee31f6dcd974b24
I20260809 19:30:20.120262 2428 raft_consensus.cc:686] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [term 1 LEADER]: Becoming Leader. State: Replica: fec4df7e43a64f64b0d94309b3b217a1, State: Running, Role: LEADER
I20260809 19:30:20.120358 2428 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } }
I20260809 19:30:20.120694 2430 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fec4df7e43a64f64b0d94309b3b217a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } } }
I20260809 19:30:20.120793 2430 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:20.120687 2432 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec4df7e43a64f64b0d94309b3b217a1. Latest consensus state: current_term: 1 leader_uuid: "fec4df7e43a64f64b0d94309b3b217a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec4df7e43a64f64b0d94309b3b217a1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43267 } } }
I20260809 19:30:20.121085 2432 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:20.121280 2445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:20.121811 2445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:20.123517 2445 catalog_manager.cc:1347] Generated new cluster ID: 81c8e5f145e04fe9b48b78140d8f1942
I20260809 19:30:20.123579 2445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:20.131426 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.120851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.31.57.254:43267)
0809 19:30:20.121269 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.121277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.121307 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:30:20.121453 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.121518 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.121583 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.121838 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.122631 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.124049 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.125068 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.125649 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.125751 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.128092 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.128095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.128096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.128106 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.129863 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.130219 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.130221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.130222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.130358 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.130551 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.130552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.130552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.130849 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.130996 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.131062 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.131186 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":86,"threads_started":1}
I20260809 19:30:20.149485 2445 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:20.149955 2445 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:20.153204 2445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Generated new TSK 0
I20260809 19:30:20.153357 2445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:20.184211 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43267
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:20.279523 2466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:20.279716 2466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:20.279749 2466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:20.280813 2466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:20.280875 2466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:20.280905 2466 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:30:20.280929 2466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:20.280964 2466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:20.282397 2466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:20.282655 2466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:20.282896 2466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:20.287533 2477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.287566 2466 server_base.cc:1034] running on GCE node
W20260809 19:30:20.287569 2476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.287541 2479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.287866 2466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:20.288085 2466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:20.289238 2466 hybrid_clock.cc:648] HybridClock initialized: now 1786303820289224 us; error 44 us; skew 500 ppm
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:20.291580 2466 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:20.292094 2466 webserver.cc:460] Webserver started at http://127.31.57.193:45737/ using document root <none> and password file <none>
I20260809 19:30:20.292342 2466 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:20.292408 2466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:20.292539 2466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:20.293573 2466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "560087126c064828bdf9c9c7399a7c32"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "a75b127c44fc5377f54074d1eedd7c58"
server_key_iv: "f7826cfa7867d4e6ab73e60731113fb1"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.293923 2466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "560087126c064828bdf9c9c7399a7c32"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "a75b127c44fc5377f54074d1eedd7c58"
server_key_iv: "f7826cfa7867d4e6ab73e60731113fb1"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.295269 2466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:20.296038 2491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.296218 2466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:20.296315 2466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "560087126c064828bdf9c9c7399a7c32"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "a75b127c44fc5377f54074d1eedd7c58"
server_key_iv: "f7826cfa7867d4e6ab73e60731113fb1"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.296396 2466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:20.306726 2466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:20.308864 2466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:20.308964 2466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:20.309405 2466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:20.309443 2466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.309464 2466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:20.309512 2466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.316090 2466 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:34619
I20260809 19:30:20.316171 2691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:34619 every 8 connection(s)
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:20.316536 2466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:20.319345 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2466
I20260809 19:30:20.319434 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.309863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:48467 (local address 127.31.57.254:43267)
0809 19:30:20.309934 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.309936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.310088 (+ 152us) server_negotiation.cc:407] Connection header received
0809 19:30:20.310504 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.310506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.310519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.310591 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.311363 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.312215 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.313830 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.314193 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.314264 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.316903 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.316904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.316905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.316912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.318333 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.318675 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.318676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.318676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.318725 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.318879 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.318880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.318880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.318962 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.319003 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.319215 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.319301 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:20.319460 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:20.319456 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.309831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.193:48467)
0809 19:30:20.309973 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.310007 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.310417 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.310641 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.310644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.310957 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.311241 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.311247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.312335 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.312336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.313722 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.313726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.314302 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.314302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.314432 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.314931 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.314941 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.316815 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.318398 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.318402 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.318404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.318591 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.318763 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.318764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.318764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.318844 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.318994 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.318996 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.319088 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.319156 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.319260 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260809 19:30:20.319659 31975 external_mini_cluster.cc:1428] Setting key 8d7138566ed6795ddf6a5efbc4f75672
I20260809 19:30:20.320362 2693 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43267
I20260809 19:30:20.320449 2693 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:20.320627 2693 heartbeater.cc:511] Master 127.31.57.254:43267 requested a full tablet report, sending...
I20260809 19:30:20.321110 2355 ts_manager.cc:194] Registered new tserver with Master: 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619)
I20260809 19:30:20.321807 2355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:48467
I20260809 19:30:20.326591 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.321246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:58695 (local address 127.31.57.254:43267)
0809 19:30:20.321290 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.321292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.321417 (+ 125us) server_negotiation.cc:407] Connection header received
0809 19:30:20.321460 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.321461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.321470 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.321523 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.321873 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.322410 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.323103 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.323262 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.323318 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.324299 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.324300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.324301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.324306 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.325597 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.325954 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.325955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.325955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.326000 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.326197 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.326197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.326197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.326281 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.326327 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.326490 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.326541 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:30:20.326694 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.321104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.193:58695)
0809 19:30:20.321335 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.321347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.321409 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.321528 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.321528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.321735 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.321796 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.321798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.322497 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.322499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.323010 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.323012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.323356 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.323357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.323465 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.323812 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.323817 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.324215 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.325671 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.325672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.325673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.325876 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.326059 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.326060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.326060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.326127 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.326347 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.326348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.326397 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.326478 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.326578 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:20.345321 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43267
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:20.422892 2717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:20.423090 2717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:20.423111 2717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:20.424117 2717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:20.424150 2717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:20.424171 2717 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:30:20.424182 2717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:20.424206 2717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:20.425561 2717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:20.425729 2717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:20.425884 2717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:20.430626 2729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.430627 2726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.430660 2725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.430951 2717 server_base.cc:1034] running on GCE node
I20260809 19:30:20.431149 2717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:20.431402 2717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:20.432554 2717 hybrid_clock.cc:648] HybridClock initialized: now 1786303820432539 us; error 31 us; skew 500 ppm
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:20.434939 2717 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:20.435344 2717 webserver.cc:460] Webserver started at http://127.31.57.194:37657/ using document root <none> and password file <none>
I20260809 19:30:20.435631 2717 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:20.435705 2717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:20.435859 2717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:20.436842 2717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "97e455b3b72a4cbbba4457a40bc104d1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "4fcdbea79cdc5edff68375b80192f072"
server_key_iv: "8b12ee5791fcd4d157a30efb3b645fe0"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.437197 2717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "97e455b3b72a4cbbba4457a40bc104d1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "4fcdbea79cdc5edff68375b80192f072"
server_key_iv: "8b12ee5791fcd4d157a30efb3b645fe0"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.438436 2717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:20.439106 2744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.439257 2717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:20.439345 2717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "97e455b3b72a4cbbba4457a40bc104d1"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "4fcdbea79cdc5edff68375b80192f072"
server_key_iv: "8b12ee5791fcd4d157a30efb3b645fe0"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.439473 2717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:20.499166 2717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:20.501262 2717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:20.501423 2717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:20.501953 2717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:20.502018 2717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.502063 2717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:20.502110 2717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:20.509254 2717 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:43541
I20260809 19:30:20.509617 2717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:20.509640 2924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:43541 every 8 connection(s)
I20260809 19:30:20.510598 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2717
I20260809 19:30:20.510704 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:20.510877 31975 external_mini_cluster.cc:1428] Setting key 65e7948db6f674f5dca95f922bb8da58
I20260809 19:30:20.512079 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.502512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.194:44901)
0809 19:30:20.502689 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.502724 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.503195 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.503508 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.503511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.503724 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.503916 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.503920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.504932 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.504933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.506200 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.506204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.506718 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.506719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.506875 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.507440 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.507449 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.508993 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.510722 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.510729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.510731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.510999 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.511224 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.511224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.511224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.511324 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.511608 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.511614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.511719 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.511803 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.511914 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260809 19:30:20.512077 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.502519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:44901 (local address 127.31.57.254:43267)
0809 19:30:20.502581 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.502583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.502800 (+ 217us) server_negotiation.cc:407] Connection header received
0809 19:30:20.503298 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.503299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.503314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.503402 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.504017 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.504809 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.506297 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.506609 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.506690 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.509101 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.509102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.509103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.509109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.510626 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.511091 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.511092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.511092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.511147 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.511405 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.511406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.511406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.511524 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.511618 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.511847 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.511950 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:30:20.512609 2926 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43267
I20260809 19:30:20.512686 2926 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:20.512835 2926 heartbeater.cc:511] Master 127.31.57.254:43267 requested a full tablet report, sending...
I20260809 19:30:20.513208 2355 ts_manager.cc:194] Registered new tserver with Master: 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541)
I20260809 19:30:20.513664 2355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:44901
I20260809 19:30:20.520612 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.514665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:48141 (local address 127.31.57.254:43267)
0809 19:30:20.514731 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.514733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.514955 (+ 222us) server_negotiation.cc:407] Connection header received
0809 19:30:20.515058 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.515059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.515069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.515100 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.515589 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.516400 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.517171 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.517360 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.517429 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.518260 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.518261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.518261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.518268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.519510 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.519827 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.519828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.519829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.519881 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.520096 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.520097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.520098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.520233 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.520304 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.520463 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.520536 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:30:20.520623 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.514716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.194:48141)
0809 19:30:20.514884 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.514896 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.514983 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.515170 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.515170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.515417 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.515478 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.515480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.516495 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.516496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.517087 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.517089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.517430 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.517431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.517508 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.517923 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.517927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.518175 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.519596 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.519597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.519598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.519757 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.519982 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.519983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.519983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.520039 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.520332 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.520333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.520374 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.520440 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.520547 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:20.535858 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43267
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:20.611600 2941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:20.611805 2941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:20.611845 2941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:20.613076 2941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:20.613162 2941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:20.613190 2941 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:30:20.613201 2941 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:20.613217 2941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:20.614656 2941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:43267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:20.614835 2941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:20.615000 2941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:20.619791 2959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.619925 2952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:20.619817 2956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:20.620005 2941 server_base.cc:1034] running on GCE node
I20260809 19:30:20.620226 2941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:20.620455 2941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:20.621619 2941 hybrid_clock.cc:648] HybridClock initialized: now 1786303820621605 us; error 34 us; skew 500 ppm
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:20.624188 2941 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:20.624588 2941 webserver.cc:460] Webserver started at http://127.31.57.195:41501/ using document root <none> and password file <none>
I20260809 19:30:20.624821 2941 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:20.624902 2941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:20.625051 2941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:20.626281 2941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "68aea2843b44d7b7770e523cc19ad228"
server_key_iv: "191e14b7af2fd9bc6d617a24deaa469c"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.626609 2941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "68aea2843b44d7b7770e523cc19ad228"
server_key_iv: "191e14b7af2fd9bc6d617a24deaa469c"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.628011 2941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 19:30:20.628737 2968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.628906 2941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:20.628996 2941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
format_stamp: "Formatted at 2026-08-09 19:30:20 on dist-test-slave-x4qh"
server_key: "68aea2843b44d7b7770e523cc19ad228"
server_key_iv: "191e14b7af2fd9bc6d617a24deaa469c"
server_key_version: "encryptionkey@0"
I20260809 19:30:20.629067 2941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:20.640432 2941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:20.642184 2941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:20.642402 2941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:20.642903 2941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:20.642963 2941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:20.643003 2941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:20.643034 2941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:20.649405 2941 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:40099
I20260809 19:30:20.649706 2941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:20.649865 3160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:40099 every 8 connection(s)
I20260809 19:30:20.650064 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2941
I20260809 19:30:20.650141 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:20.650309 31975 external_mini_cluster.cc:1428] Setting key 428488ae116efd9d5d247816ebb0f802
I20260809 19:30:20.652514 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.643324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:40331 (local address 127.31.57.254:43267)
0809 19:30:20.643412 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.643414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.643573 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:30:20.643987 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.643988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.644002 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.644085 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.644749 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.645561 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.647005 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.647321 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.647418 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.649789 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.649790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.649791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.649798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.651485 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.651842 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.651843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.651843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.651892 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.652060 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.652061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.652061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.652155 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.652208 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.652398 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.652455 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 19:30:20.652773 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.643268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.195:40331)
0809 19:30:20.643435 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.643481 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.643886 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.644127 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.644130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.644397 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.644648 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.644653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.645679 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.645680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.646907 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.646911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.647449 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.647450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.647581 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.648033 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.648064 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.649694 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.651565 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.651571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.651572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.651771 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.651934 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.651934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.651934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.652013 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.652202 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.652205 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.652299 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.652369 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.652473 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260809 19:30:20.653213 3162 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43267
I20260809 19:30:20.653294 3162 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:20.653465 3162 heartbeater.cc:511] Master 127.31.57.254:43267 requested a full tablet report, sending...
I20260809 19:30:20.653870 2355 ts_manager.cc:194] Registered new tserver with Master: e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099)
I20260809 19:30:20.654182 2355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:40331
I20260809 19:30:20.658787 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.654235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:56535 (local address 127.31.57.254:43267)
0809 19:30:20.654314 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.654316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.654351 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:30:20.654407 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.654407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.654416 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.654486 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.654767 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.655237 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.655939 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.656106 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.656159 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.656909 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.656910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.656910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.656915 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.658120 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.658360 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.658360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.658361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.658395 (+ 34us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.658498 (+ 103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.658498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.658499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.658564 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.658599 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.658679 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.658723 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 19:30:20.658795 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.654172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.31.57.195:56535)
0809 19:30:20.654299 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.654308 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.654364 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.654491 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.654491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.654639 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:20.654691 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.654693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.655325 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.655326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.655869 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.655871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.656170 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.656170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.656225 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.656555 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:20.656559 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:20.656840 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:20.658180 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.658181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.658181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.658313 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.658430 (+ 117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:20.658430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:20.658431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:20.658466 (+ 35us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:20.658593 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:20.658593 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:20.658620 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.658659 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.658715 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
I20260809 19:30:20.662164 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:30:20 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:20.683166 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.677103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.31.57.254:43267)
0809 19:30:20.677161 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.677163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.677171 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:20.677239 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.677240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.677248 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.677321 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:20.677831 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.678232 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.678884 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.679049 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.679119 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.680990 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:20.680991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:20.680991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:20.680998 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:20.682035 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.682397 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.682398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.682398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.682456 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:20.682650 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:20.682651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:20.682651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:20.682761 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:20.682859 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.682990 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.683064 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:30:20.684664 2355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:20.686139 2355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:20.692229 3189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:34619 (local address 127.0.0.1:46030)
0809 19:30:20.688763 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.688769 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.688828 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.689015 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.689016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.689018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:20.689061 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.689062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.689962 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.689963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.691038 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.691040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.691897 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.691898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.691992 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.692005 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.692103 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.692159 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":76,"threads_started":1}
I20260809 19:30:20.692456 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.31.57.193:34619)
0809 19:30:20.688648 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.688653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.688826 (+ 173us) server_negotiation.cc:407] Connection header received
0809 19:30:20.688893 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.688895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.688909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.689006 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:20.689150 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.689870 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.691104 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.691774 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.691891 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.691965 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.692121 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.692173 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260809 19:30:20.692691 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.31.57.194:43541)
0809 19:30:20.688729 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.688733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.688742 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:20.688779 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.688781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.688796 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.688862 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:20.689079 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.690141 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.691428 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.692140 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.692208 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.692254 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.692563 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.692603 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260809 19:30:20.692845 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43541 (local address 127.0.0.1:57890)
0809 19:30:20.688663 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.688671 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.688703 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.688921 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.688924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.688925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:20.689003 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.689004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.690229 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.690230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.691115 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.691117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.692289 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.692290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.692350 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.692354 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.692403 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.692439 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":10,"thread_start_us":71,"threads_started":1}
I20260809 19:30:20.693104 2582 tablet_service.cc:1467] Processing CreateTablet for tablet c0a492b9707441a0b866a9561a6246f4 (DEFAULT_TABLE table=test-table [id=3d1338a4dac9493ab85eb049393879f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:20.693426 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.0.0.1:53610)
0809 19:30:20.688441 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:20.688478 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:20.688559 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:20.688920 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:20.688923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:20.688925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:20.688994 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.688996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.691184 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.691185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.692319 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:20.692320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.693199 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.693200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:20.693268 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.693271 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:20.693325 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:30:20.693360 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":52,"threads_started":1}
I20260809 19:30:20.693440 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:20.688421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.31.57.195:40099)
0809 19:30:20.688693 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:20.688697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:20.688708 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:30:20.688747 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:20.688750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:20.688764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:20.688849 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:20.689184 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.690134 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.692404 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:20.693115 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:20.693196 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:20.693242 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:20.693338 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:30:20.693367 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
I20260809 19:30:20.693476 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a492b9707441a0b866a9561a6246f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:20.693487 2811 tablet_service.cc:1467] Processing CreateTablet for tablet c0a492b9707441a0b866a9561a6246f4 (DEFAULT_TABLE table=test-table [id=3d1338a4dac9493ab85eb049393879f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:20.693794 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a492b9707441a0b866a9561a6246f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:20.695894 3043 tablet_service.cc:1467] Processing CreateTablet for tablet c0a492b9707441a0b866a9561a6246f4 (DEFAULT_TABLE table=test-table [id=3d1338a4dac9493ab85eb049393879f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:20.696097 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a492b9707441a0b866a9561a6246f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:20.696997 3195 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Bootstrap starting.
I20260809 19:30:20.697599 3195 tablet_bootstrap.cc:654] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:20.697965 3195 log.cc:824] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:20.698352 3194 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Bootstrap starting.
I20260809 19:30:20.698824 3201 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Bootstrap starting.
I20260809 19:30:20.698994 3194 tablet_bootstrap.cc:654] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:20.698995 3195 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: No bootstrap required, opened a new log
I20260809 19:30:20.699088 3195 ts_tablet_manager.cc:1397] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:20.699348 3194 log.cc:824] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:20.700021 3201 tablet_bootstrap.cc:654] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:20.700096 3194 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: No bootstrap required, opened a new log
I20260809 19:30:20.700165 3194 ts_tablet_manager.cc:1397] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:20.700397 3201 log.cc:824] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:20.701207 3201 tablet_bootstrap.cc:492] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: No bootstrap required, opened a new log
I20260809 19:30:20.701285 3201 ts_tablet_manager.cc:1397] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:30:20.701424 3195 raft_consensus.cc:348] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.701601 3195 raft_consensus.cc:374] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:20.701651 3195 raft_consensus.cc:729] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Initialized, Role: FOLLOWER
I20260809 19:30:20.701776 3195 consensus_queue.cc:260] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.702044 3195 ts_tablet_manager.cc:1428] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:30:20.701958 3194 raft_consensus.cc:348] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.702101 3194 raft_consensus.cc:374] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:20.702134 3194 raft_consensus.cc:729] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Initialized, Role: FOLLOWER
I20260809 19:30:20.702159 2926 heartbeater.cc:503] Master 127.31.57.254:43267 was elected leader, sending a full tablet report...
I20260809 19:30:20.702265 3194 consensus_queue.cc:260] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.702543 3194 ts_tablet_manager.cc:1428] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:30:20.702678 2693 heartbeater.cc:503] Master 127.31.57.254:43267 was elected leader, sending a full tablet report...
I20260809 19:30:20.703517 3201 raft_consensus.cc:348] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.703636 3201 raft_consensus.cc:374] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:20.703660 3201 raft_consensus.cc:729] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Initialized, Role: FOLLOWER
I20260809 19:30:20.703728 3201 consensus_queue.cc:260] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:20.703918 3201 ts_tablet_manager.cc:1428] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:30:20.703963 3162 heartbeater.cc:503] Master 127.31.57.254:43267 was elected leader, sending a full tablet report...
W20260809 19:30:20.760515 2928 tablet.cc:2369] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:20.817255 2697 tablet.cc:2369] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:20.900748 3164 tablet.cc:2369] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:21.020902 3205 raft_consensus.cc:482] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:21.021018 3205 raft_consensus.cc:504] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:21.021389 3205 leader_election.cc:288] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099), 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619)
I20260809 19:30:21.024545 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.021672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:50289 (local address 127.31.57.193:34619)
0809 19:30:21.021723 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.021726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.021815 (+ 89us) server_negotiation.cc:407] Connection header received
0809 19:30:21.021880 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.021881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.021892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.021948 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:21.022110 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.022619 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.023757 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.024169 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.024227 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.024233 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.024421 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.024463 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:21.024566 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.021588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:34619 (local address 127.31.57.194:50289)
0809 19:30:21.021712 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.021723 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.021813 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.021972 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.021972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.021975 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:21.022041 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.022043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.022691 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.022692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.023671 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.023674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.024249 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.024250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.024334 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.024338 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.024411 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.024453 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":9,"thread_start_us":73,"threads_started":1}
I20260809 19:30:21.024644 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.021740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:35025 (local address 127.31.57.195:40099)
0809 19:30:21.021813 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.021816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.021887 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:30:21.021968 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.021969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.021978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.022043 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:21.022204 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.022814 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.023877 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.024263 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.024335 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.024355 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.024525 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.024572 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:30:21.024724 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.021604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.31.57.194:35025)
0809 19:30:21.021815 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.021824 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.021883 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.022049 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.022050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.022056 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:21.022139 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.022141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.022895 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.022896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.023793 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.023795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.024354 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.024355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.024443 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.024447 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.024506 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.024543 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260809 19:30:21.025039 3082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a492b9707441a0b866a9561a6246f4" candidate_uuid: "97e455b3b72a4cbbba4457a40bc104d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" is_pre_election: true
I20260809 19:30:21.025215 3082 raft_consensus.cc:2454] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e455b3b72a4cbbba4457a40bc104d1 in term 0.
I20260809 19:30:21.025205 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a492b9707441a0b866a9561a6246f4" candidate_uuid: "97e455b3b72a4cbbba4457a40bc104d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32" is_pre_election: true
I20260809 19:30:21.025345 2607 raft_consensus.cc:2454] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e455b3b72a4cbbba4457a40bc104d1 in term 0.
I20260809 19:30:21.025449 2747 leader_election.cc:302] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e455b3b72a4cbbba4457a40bc104d1, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:21.025591 3205 raft_consensus.cc:2781] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:21.025679 3205 raft_consensus.cc:482] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:21.025719 3205 raft_consensus.cc:3037] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.026472 3205 raft_consensus.cc:504] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:21.026623 3205 leader_election.cc:288] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [CANDIDATE]: Term 1 election: Requested vote from peers e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099), 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619)
I20260809 19:30:21.026789 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a492b9707441a0b866a9561a6246f4" candidate_uuid: "97e455b3b72a4cbbba4457a40bc104d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32"
I20260809 19:30:21.026863 2607 raft_consensus.cc:3037] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.027459 3082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a492b9707441a0b866a9561a6246f4" candidate_uuid: "97e455b3b72a4cbbba4457a40bc104d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
I20260809 19:30:21.027542 3082 raft_consensus.cc:3037] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.027716 2607 raft_consensus.cc:2454] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e455b3b72a4cbbba4457a40bc104d1 in term 1.
I20260809 19:30:21.027885 2748 leader_election.cc:302] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, 97e455b3b72a4cbbba4457a40bc104d1; no voters:
I20260809 19:30:21.027978 3205 raft_consensus.cc:2781] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:21.028111 3205 raft_consensus.cc:686] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 LEADER]: Becoming Leader. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Running, Role: LEADER
I20260809 19:30:21.028196 3205 consensus_queue.cc:237] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } }
I20260809 19:30:21.028451 3082 raft_consensus.cc:2454] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e455b3b72a4cbbba4457a40bc104d1 in term 1.
I20260809 19:30:21.028954 2355 catalog_manager.cc:5495] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194). New cstate: current_term: 1 leader_uuid: "97e455b3b72a4cbbba4457a40bc104d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } health_report { overall_health: HEALTHY } } }
Aug 09 19:30:21 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303820, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:21.066064 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.059884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43267 (local address 127.0.0.1:45054)
0809 19:30:21.060018 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.060027 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.060081 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.060190 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.060191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.060502 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:21.060543 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.060545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.061089 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.061089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.061556 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.061557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.061862 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.061863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.061901 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.062352 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:21.062356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:21.063671 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:21.065064 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:21.065071 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:21.065073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:21.065249 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:21.065459 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:21.065460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:21.065460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:21.065516 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:21.065740 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:21.065747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:21.065786 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.065864 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.065962 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260809 19:30:21.066078 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.059955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.31.57.254:43267)
0809 19:30:21.060034 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.060036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.060084 (+ 48us) server_negotiation.cc:407] Connection header received
0809 19:30:21.060117 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.060117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.060125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.060169 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:21.060606 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.061029 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.061621 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.061775 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.061846 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.063759 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:21.063760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:21.063761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:21.063767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:21.064983 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:21.065339 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:21.065340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:21.065341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:21.065393 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:21.065591 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:21.065591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:21.065592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:21.065682 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:21.065748 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.065885 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.066007 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":50}
I20260809 19:30:21.067898 2355 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:45054:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:30:21.068112 2355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:21.069684 2582 tablet_service.cc:1467] Processing CreateTablet for tablet 6b6b85561d00404aa51a1f7ff7edc6c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0f023d02e7d446095089172a0e1d7ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:21.069684 2811 tablet_service.cc:1467] Processing CreateTablet for tablet 6b6b85561d00404aa51a1f7ff7edc6c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0f023d02e7d446095089172a0e1d7ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:21.069715 3043 tablet_service.cc:1467] Processing CreateTablet for tablet 6b6b85561d00404aa51a1f7ff7edc6c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0f023d02e7d446095089172a0e1d7ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:21.069866 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b6b85561d00404aa51a1f7ff7edc6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.069893 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b6b85561d00404aa51a1f7ff7edc6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.069921 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b6b85561d00404aa51a1f7ff7edc6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.070917 3201 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: Bootstrap starting.
I20260809 19:30:21.071332 3195 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: Bootstrap starting.
I20260809 19:30:21.071336 3194 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: Bootstrap starting.
I20260809 19:30:21.071471 3201 tablet_bootstrap.cc:654] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.071708 3195 tablet_bootstrap.cc:654] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.071738 3194 tablet_bootstrap.cc:654] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.072265 3201 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: No bootstrap required, opened a new log
I20260809 19:30:21.072343 3201 ts_tablet_manager.cc:1397] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:21.072499 3201 raft_consensus.cc:348] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.072659 3201 raft_consensus.cc:374] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.072683 3201 raft_consensus.cc:729] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.072726 3201 consensus_queue.cc:260] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.072829 3201 ts_tablet_manager.cc:1428] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:21.072854 3195 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: No bootstrap required, opened a new log
I20260809 19:30:21.072909 3195 ts_tablet_manager.cc:1397] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:21.072882 3207 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.072978 3207 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.073055 3195 raft_consensus.cc:348] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.073148 3195 raft_consensus.cc:374] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.073180 3195 raft_consensus.cc:729] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.073220 3195 consensus_queue.cc:260] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.073240 3194 tablet_bootstrap.cc:492] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: No bootstrap required, opened a new log
I20260809 19:30:21.073283 3194 ts_tablet_manager.cc:1397] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:21.073311 3195 ts_tablet_manager.cc:1428] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:21.073427 3194 raft_consensus.cc:348] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.073426 3205 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.073509 3194 raft_consensus.cc:374] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.073510 3205 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.073534 3194 raft_consensus.cc:729] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.073596 3194 consensus_queue.cc:260] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.073695 3194 ts_tablet_manager.cc:1428] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:21.073738 3206 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.073796 3206 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.086114 3207 raft_consensus.cc:482] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:21.086163 3207 raft_consensus.cc:504] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.086454 3207 leader_election.cc:288] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619), 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541)
I20260809 19:30:21.088876 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.086613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:53189 (local address 127.31.57.193:34619)
0809 19:30:21.086646 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.086648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.086664 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:30:21.086686 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.086686 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.086694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.086741 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:21.086921 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.087355 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088246 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.088559 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088635 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.088643 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.088791 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.088823 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:21.088877 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.086553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:34619 (local address 127.31.57.195:53189)
0809 19:30:21.086595 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.086605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.086642 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.086767 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.086768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.086769 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:21.086831 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.086833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.087491 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.087492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.088151 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.088153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.088641 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.088718 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.088720 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.088784 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.088820 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:30:21.089103 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.086792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:47649 (local address 127.31.57.194:43541)
0809 19:30:21.086861 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.086864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.086871 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:21.086929 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.086930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.086939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.086999 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:21.087154 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.087611 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088409 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.088722 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088815 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.088822 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.088984 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.089031 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 19:30:21.089104 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.086598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43541 (local address 127.31.57.195:47649)
0809 19:30:21.086814 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.086824 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.086863 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.087002 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.087002 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.087003 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:21.087083 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.087085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.087684 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.087685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.088318 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.088320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.088843 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.088845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.088903 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.088905 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.088973 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.089016 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260809 19:30:21.089107 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b6b85561d00404aa51a1f7ff7edc6c0" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32" is_pre_election: true
I20260809 19:30:21.089169 2607 raft_consensus.cc:2454] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 0.
I20260809 19:30:21.089330 2972 leader_election.cc:302] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:21.089358 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b6b85561d00404aa51a1f7ff7edc6c0" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1" is_pre_election: true
I20260809 19:30:21.089437 2844 raft_consensus.cc:2454] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 0.
I20260809 19:30:21.089452 3207 raft_consensus.cc:2781] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:21.089480 3207 raft_consensus.cc:482] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:21.089511 3207 raft_consensus.cc:3037] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.090078 3207 raft_consensus.cc:504] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.090229 3207 leader_election.cc:288] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 election: Requested vote from peers 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619), 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541)
I20260809 19:30:21.090366 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b6b85561d00404aa51a1f7ff7edc6c0" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32"
I20260809 19:30:21.090440 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b6b85561d00404aa51a1f7ff7edc6c0" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1"
I20260809 19:30:21.090485 2607 raft_consensus.cc:3037] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.090513 2844 raft_consensus.cc:3037] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.091184 2607 raft_consensus.cc:2454] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 1.
I20260809 19:30:21.091272 2844 raft_consensus.cc:2454] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 1.
I20260809 19:30:21.091413 2972 leader_election.cc:302] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:21.091572 3207 raft_consensus.cc:2781] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:21.091749 3207 raft_consensus.cc:686] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 LEADER]: Becoming Leader. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Running, Role: LEADER
I20260809 19:30:21.091866 3207 consensus_queue.cc:237] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.092262 3228 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: TxnStatusTablet state changed. Reason: New leader e1ae66206fb14bfc820fd59ed6921d9e. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.092358 3228 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:21.092572 2355 catalog_manager.cc:5495] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195). New cstate: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } health_report { overall_health: HEALTHY } } }
I20260809 19:30:21.092869 3231 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:30:21.092988 3231 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:30:21.103281 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.100712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.31.57.195:40099)
0809 19:30:21.100766 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.100768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.100775 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:21.100798 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.100798 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.100811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.100865 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.101003 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.101439 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.102523 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.102759 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.102811 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.103015 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.103095 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.103163 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.103226 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.103242 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:21.103339 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.100469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.0.0.1:53612)
0809 19:30:21.100551 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.100563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.100613 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.100896 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.100897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.100898 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:21.100935 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.100937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.101507 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.101508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.102405 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.102408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.102848 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.102849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.102913 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.102929 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.103165 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.103173 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.103226 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.103266 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 19:30:21.104909 2844 raft_consensus.cc:1261] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Refusing update from remote peer e1ae66206fb14bfc820fd59ed6921d9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.104979 2607 raft_consensus.cc:1261] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Refusing update from remote peer e1ae66206fb14bfc820fd59ed6921d9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.105242 3207 consensus_queue.cc:1035] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.105384 3228 consensus_queue.cc:1035] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.106349 3206 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: TxnStatusTablet state changed. Reason: New leader e1ae66206fb14bfc820fd59ed6921d9e. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.106432 3206 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.106492 3205 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: TxnStatusTablet state changed. Reason: New leader e1ae66206fb14bfc820fd59ed6921d9e. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.106562 3205 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.107239 3228 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.107323 3228 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:21.107947 3207 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.108023 3207 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P e1ae66206fb14bfc820fd59ed6921d9e: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:21.108107 3236 mvcc.cc:205] Tried to move back new op lower bound from 7316700451243511808 to 7316700451192770560. Current Snapshot: MvccSnapshot[applied={T|T < 7316700451243511808}]
I20260809 19:30:21.108177 3206 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.108245 3206 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 560087126c064828bdf9c9c7399a7c32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.108628 3205 tablet_replica.cc:439] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } } }
I20260809 19:30:21.108727 3205 tablet_replica.cc:442] T 6b6b85561d00404aa51a1f7ff7edc6c0 P 97e455b3b72a4cbbba4457a40bc104d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:21.112162 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.109624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.31.57.194:43541)
0809 19:30:21.109655 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.109657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.109686 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:30:21.109741 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.109741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.109755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.109814 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.109962 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.110348 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.111335 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.111582 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.111631 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.111805 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.111933 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.112024 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.112081 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.112101 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:30:21.116878 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.114623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:52893 (local address 127.31.57.195:40099)
0809 19:30:21.114654 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.114655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.114683 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:30:21.114738 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.114738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.114747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.114792 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.114900 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.115258 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.116315 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.116467 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.116513 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.116711 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.116733 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.116765 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.116826 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.116839 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:30:21.117079 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.114478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.31.57.194:52893)
0809 19:30:21.114629 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.114639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.114695 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.114803 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.114804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.114805 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:21.114853 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.114854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.115328 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.115329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.116223 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.116225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.116541 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.116542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.116606 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.116623 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.116779 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.116780 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.116824 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.116860 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 19:30:21.119091 3082 raft_consensus.cc:1261] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Refusing update from remote peer 97e455b3b72a4cbbba4457a40bc104d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.119102 2607 raft_consensus.cc:1261] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Refusing update from remote peer 97e455b3b72a4cbbba4457a40bc104d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.119365 3210 consensus_queue.cc:1035] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.119484 3205 consensus_queue.cc:1035] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.126979 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.124764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:57565 (local address 127.31.57.194:43541)
0809 19:30:21.124812 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.124813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.124820 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:21.124837 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.124837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.124844 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.124909 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.125030 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.125422 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.126228 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.126523 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.126580 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.126757 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.126786 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.126831 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.126895 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.126912 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:30:21.126986 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.124589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43541 (local address 127.31.57.195:57565)
0809 19:30:21.124661 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.124671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.124713 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.124913 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.124913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.124914 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:21.124966 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.124967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.125506 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.125507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.126101 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.126103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.126580 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.126581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.126642 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.126666 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.126831 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.126832 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.126892 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.126928 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:30:21.137018 2356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:21.137104 2356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:21.137828 2356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Sending DeleteTablet for 3 replicas of tablet c0a492b9707441a0b866a9561a6246f4
I20260809 19:30:21.138100 2811 tablet_service.cc:1514] Processing DeleteTablet for tablet c0a492b9707441a0b866a9561a6246f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:21 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:57890
I20260809 19:30:21.138113 2582 tablet_service.cc:1514] Processing DeleteTablet for tablet c0a492b9707441a0b866a9561a6246f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:21 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260809 19:30:21.138375 3265 tablet_replica.cc:330] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: stopping tablet replica
I20260809 19:30:21.138407 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet c0a492b9707441a0b866a9561a6246f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:21 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53610
I20260809 19:30:21.138535 3265 raft_consensus.cc:2229] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:21.138646 3265 raft_consensus.cc:2258] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:21.138777 2356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:21.138904 2356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:21.139147 3269 tablet_replica.cc:330] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: stopping tablet replica
I20260809 19:30:21.139236 3269 raft_consensus.cc:2229] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:21.139294 3269 raft_consensus.cc:2258] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:21.139509 3267 tablet_replica.cc:330] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: stopping tablet replica
I20260809 19:30:21.139613 3267 raft_consensus.cc:2229] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:21.139671 3267 raft_consensus.cc:2258] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:21.139911 3269 ts_tablet_manager.cc:1905] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:21.140043 3267 ts_tablet_manager.cc:1905] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:21.140492 2582 tablet_service.cc:1467] Processing CreateTablet for tablet 240fad98e2034f808cf7346a9c56b918 (DEFAULT_TABLE table=test-table [id=f14f9c4445c349f09a1318df8679eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:21.140519 2811 tablet_service.cc:1467] Processing CreateTablet for tablet 240fad98e2034f808cf7346a9c56b918 (DEFAULT_TABLE table=test-table [id=f14f9c4445c349f09a1318df8679eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:21.140597 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240fad98e2034f808cf7346a9c56b918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.140626 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240fad98e2034f808cf7346a9c56b918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.141660 3269 ts_tablet_manager.cc:1918] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:21.142095 3195 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Bootstrap starting.
I20260809 19:30:21.142619 3195 tablet_bootstrap.cc:654] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.142647 3265 ts_tablet_manager.cc:1905] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:21.142915 3269 log.cc:1196] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/c0a492b9707441a0b866a9561a6246f4
I20260809 19:30:21.142939 3194 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Bootstrap starting.
I20260809 19:30:21.143299 3269 ts_tablet_manager.cc:1939] T c0a492b9707441a0b866a9561a6246f4 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting consensus metadata
I20260809 19:30:21.143314 3194 tablet_bootstrap.cc:654] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.143517 3043 tablet_service.cc:1467] Processing CreateTablet for tablet 240fad98e2034f808cf7346a9c56b918 (DEFAULT_TABLE table=test-table [id=f14f9c4445c349f09a1318df8679eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:21.143616 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240fad98e2034f808cf7346a9c56b918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:21.144770 3201 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Bootstrap starting.
I20260809 19:30:21.144938 3265 ts_tablet_manager.cc:1918] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:21.145042 3265 log.cc:1196] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/c0a492b9707441a0b866a9561a6246f4
I20260809 19:30:21.145179 3201 tablet_bootstrap.cc:654] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:21.145411 3265 ts_tablet_manager.cc:1939] T c0a492b9707441a0b866a9561a6246f4 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting consensus metadata
I20260809 19:30:21.145612 2332 catalog_manager.cc:4841] TS e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099): tablet c0a492b9707441a0b866a9561a6246f4 (table test-table [id=3d1338a4dac9493ab85eb049393879f8]) successfully deleted
I20260809 19:30:21.145845 2330 catalog_manager.cc:4841] TS 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541): tablet c0a492b9707441a0b866a9561a6246f4 (table test-table [id=3d1338a4dac9493ab85eb049393879f8]) successfully deleted
I20260809 19:30:21.147534 3267 ts_tablet_manager.cc:1918] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:21.147605 3267 log.cc:1196] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/c0a492b9707441a0b866a9561a6246f4
I20260809 19:30:21.147853 3267 ts_tablet_manager.cc:1939] T c0a492b9707441a0b866a9561a6246f4 P 560087126c064828bdf9c9c7399a7c32: Deleting consensus metadata
I20260809 19:30:21.148259 2335 catalog_manager.cc:4841] TS 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619): tablet c0a492b9707441a0b866a9561a6246f4 (table test-table [id=3d1338a4dac9493ab85eb049393879f8]) successfully deleted
I20260809 19:30:21.148699 3201 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: No bootstrap required, opened a new log
I20260809 19:30:21.148699 3195 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: No bootstrap required, opened a new log
I20260809 19:30:21.148746 3201 ts_tablet_manager.cc:1397] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 19:30:21.148746 3195 ts_tablet_manager.cc:1397] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 19:30:21.148864 3201 raft_consensus.cc:348] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.148873 3195 raft_consensus.cc:348] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.148916 3201 raft_consensus.cc:374] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.148928 3195 raft_consensus.cc:374] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.148936 3201 raft_consensus.cc:729] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.148948 3195 raft_consensus.cc:729] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.148972 3201 consensus_queue.cc:260] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.148989 3195 consensus_queue.cc:260] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.149078 3201 ts_tablet_manager.cc:1428] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:21.149080 3195 ts_tablet_manager.cc:1428] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:21.150112 3194 tablet_bootstrap.cc:492] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: No bootstrap required, opened a new log
I20260809 19:30:21.150157 3194 ts_tablet_manager.cc:1397] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 19:30:21.150272 3194 raft_consensus.cc:348] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.150336 3194 raft_consensus.cc:374] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:21.150353 3194 raft_consensus.cc:729] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Initialized, Role: FOLLOWER
I20260809 19:30:21.150393 3194 consensus_queue.cc:260] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.150485 3194 ts_tablet_manager.cc:1428] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:21.175791 3256 raft_consensus.cc:482] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:21.175864 3256 raft_consensus.cc:504] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.175984 3256 leader_election.cc:288] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619), 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541)
I20260809 19:30:21.176172 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240fad98e2034f808cf7346a9c56b918" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32" is_pre_election: true
I20260809 19:30:21.176170 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240fad98e2034f808cf7346a9c56b918" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1" is_pre_election: true
I20260809 19:30:21.176242 2607 raft_consensus.cc:2454] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 0.
I20260809 19:30:21.176254 2844 raft_consensus.cc:2454] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 0.
I20260809 19:30:21.176371 2972 leader_election.cc:302] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:21.176450 3256 raft_consensus.cc:2781] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:21.176478 3256 raft_consensus.cc:482] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:21.176528 3256 raft_consensus.cc:3037] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.177338 3256 raft_consensus.cc:504] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.177456 3256 leader_election.cc:288] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 election: Requested vote from peers 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619), 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541)
I20260809 19:30:21.177620 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240fad98e2034f808cf7346a9c56b918" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1"
I20260809 19:30:21.177693 2844 raft_consensus.cc:3037] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.177667 2607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240fad98e2034f808cf7346a9c56b918" candidate_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560087126c064828bdf9c9c7399a7c32"
I20260809 19:30:21.177732 2607 raft_consensus.cc:3037] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:21.178341 2844 raft_consensus.cc:2454] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 1.
I20260809 19:30:21.178454 2607 raft_consensus.cc:2454] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ae66206fb14bfc820fd59ed6921d9e in term 1.
I20260809 19:30:21.178503 2969 leader_election.cc:302] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e455b3b72a4cbbba4457a40bc104d1, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:21.178567 3256 raft_consensus.cc:2781] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:21.178604 3256 raft_consensus.cc:686] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 LEADER]: Becoming Leader. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Running, Role: LEADER
I20260809 19:30:21.178668 3256 consensus_queue.cc:237] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:21.179073 2355 catalog_manager.cc:5495] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195). New cstate: current_term: 1 leader_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } health_report { overall_health: HEALTHY } } }
I20260809 19:30:21.193410 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.190371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.31.57.195:40099)
0809 19:30:21.190412 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.190414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.190421 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:21.190443 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.190443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.190452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.190501 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.190687 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.191438 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.192515 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.192806 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.192865 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.193032 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.193064 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.193107 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.193329 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.193347 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:30:21.198628 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.195479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.31.57.195:38267)
0809 19:30:21.195535 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:21.195547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:21.195591 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:21.195868 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:21.195869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:21.195871 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:21.195909 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.195911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.196829 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.196830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.197627 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:21.197630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.198159 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.198160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:21.198217 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.198222 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.198448 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.198449 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:21.198511 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:30:21.198561 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:30:21.198845 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:21.195690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:38267 (local address 127.31.57.195:40099)
0809 19:30:21.195752 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:21.195754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:21.195762 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:21.195782 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:21.195783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:21.195791 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:21.195851 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:21.195982 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.196652 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.197780 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:21.198056 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:21.198147 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:21.198344 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:21.198375 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:21.198434 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:21.198785 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 19:30:21.198799 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:30:21.200867 2607 raft_consensus.cc:1261] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Refusing update from remote peer e1ae66206fb14bfc820fd59ed6921d9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.201128 3228 consensus_queue.cc:1035] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.202526 3234 mvcc.cc:205] Tried to move back new op lower bound from 7316700451637673984 to 7316700451548192768. Current Snapshot: MvccSnapshot[applied={T|T < 7316700451637673984}]
I20260809 19:30:21.203037 2844 raft_consensus.cc:1261] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Refusing update from remote peer e1ae66206fb14bfc820fd59ed6921d9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:21.203222 3207 consensus_queue.cc:1035] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:21.203846 3235 mvcc.cc:205] Tried to move back new op lower bound from 7316700451637673984 to 7316700451548192768. Current Snapshot: MvccSnapshot[applied={T|T < 7316700451637673984}]
W20260809 19:30:21.208133 3234 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:30:22.728466 2355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:22.728559 2355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:22.729301 2355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Sending DeleteTablet for 3 replicas of tablet 240fad98e2034f808cf7346a9c56b918
I20260809 19:30:22.729525 2582 tablet_service.cc:1514] Processing DeleteTablet for tablet 240fad98e2034f808cf7346a9c56b918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260809 19:30:22.729547 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet 240fad98e2034f808cf7346a9c56b918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53610
I20260809 19:30:22.729730 2811 tablet_service.cc:1514] Processing DeleteTablet for tablet 240fad98e2034f808cf7346a9c56b918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:57890
I20260809 19:30:22.729911 3315 tablet_replica.cc:330] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: stopping tablet replica
I20260809 19:30:22.729964 3315 raft_consensus.cc:2229] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:22.729997 3315 raft_consensus.cc:2258] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.730224 3315 ts_tablet_manager.cc:1905] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.732124 3315 ts_tablet_manager.cc:1918] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:30:22.732216 3315 log.cc:1196] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/240fad98e2034f808cf7346a9c56b918
I20260809 19:30:22.732512 3315 ts_tablet_manager.cc:1939] T 240fad98e2034f808cf7346a9c56b918 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting consensus metadata
I20260809 19:30:22.732846 3311 tablet_replica.cc:330] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: stopping tablet replica
I20260809 19:30:22.732877 2330 catalog_manager.cc:4841] TS 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541): tablet 240fad98e2034f808cf7346a9c56b918 (table test-table [id=f14f9c4445c349f09a1318df8679eb97]) successfully deleted
I20260809 19:30:22.732915 3311 raft_consensus.cc:2229] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:22.732952 3311 raft_consensus.cc:2258] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.733042 3313 tablet_replica.cc:330] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: stopping tablet replica
I20260809 19:30:22.733101 3313 raft_consensus.cc:2229] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:22.733206 3313 raft_consensus.cc:2258] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.733481 3311 ts_tablet_manager.cc:1905] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.733481 3313 ts_tablet_manager.cc:1905] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.734988 3311 ts_tablet_manager.cc:1918] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:30:22.734988 3313 ts_tablet_manager.cc:1918] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:30:22.735050 3311 log.cc:1196] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/240fad98e2034f808cf7346a9c56b918
I20260809 19:30:22.735050 3313 log.cc:1196] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/240fad98e2034f808cf7346a9c56b918
I20260809 19:30:22.735270 3311 ts_tablet_manager.cc:1939] T 240fad98e2034f808cf7346a9c56b918 P 560087126c064828bdf9c9c7399a7c32: Deleting consensus metadata
I20260809 19:30:22.735303 3313 ts_tablet_manager.cc:1939] T 240fad98e2034f808cf7346a9c56b918 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting consensus metadata
I20260809 19:30:22.735608 2335 catalog_manager.cc:4841] TS 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619): tablet 240fad98e2034f808cf7346a9c56b918 (table test-table [id=f14f9c4445c349f09a1318df8679eb97]) successfully deleted
I20260809 19:30:22.735625 2332 catalog_manager.cc:4841] TS e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099): tablet 240fad98e2034f808cf7346a9c56b918 (table test-table [id=f14f9c4445c349f09a1318df8679eb97]) successfully deleted
Aug 09 19:30:22 dist-test-slave-x4qh krb5kdc[2282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303822, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:30:22 dist-test-slave-x4qh krb5kdc[2282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303822, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:22.756793 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.746750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.31.57.254:43267)
0809 19:30:22.746881 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.746883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.746896 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:30:22.746936 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.746937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.746946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.747003 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:22.747599 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.748063 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.748752 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.748929 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.748995 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.754608 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:22.754609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:22.754610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:22.754616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:22.755949 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:22.756226 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:22.756227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:22.756227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:22.756276 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:22.756420 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:22.756420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:22.756420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:22.756504 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:22.756569 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.756650 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.756705 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260809 19:30:22.758083 2355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:22.758256 2355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:22.759581 2582 tablet_service.cc:1467] Processing CreateTablet for tablet e131e45f3e544153a45f9e92b0279827 (DEFAULT_TABLE table=test-table [id=c9db476337fd4dd6b02466523dba14ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.759581 2811 tablet_service.cc:1467] Processing CreateTablet for tablet e131e45f3e544153a45f9e92b0279827 (DEFAULT_TABLE table=test-table [id=c9db476337fd4dd6b02466523dba14ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.759605 3043 tablet_service.cc:1467] Processing CreateTablet for tablet e131e45f3e544153a45f9e92b0279827 (DEFAULT_TABLE table=test-table [id=c9db476337fd4dd6b02466523dba14ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.759718 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e131e45f3e544153a45f9e92b0279827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.759717 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e131e45f3e544153a45f9e92b0279827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.759752 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e131e45f3e544153a45f9e92b0279827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.760716 3328 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Bootstrap starting.
I20260809 19:30:22.761127 3329 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Bootstrap starting.
I20260809 19:30:22.761163 3328 tablet_bootstrap.cc:654] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.761539 3329 tablet_bootstrap.cc:654] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.761862 3328 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: No bootstrap required, opened a new log
I20260809 19:30:22.761865 3332 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Bootstrap starting.
I20260809 19:30:22.761917 3328 ts_tablet_manager.cc:1397] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:22.762022 3328 raft_consensus.cc:348] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.762113 3328 raft_consensus.cc:374] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.762113 3329 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: No bootstrap required, opened a new log
I20260809 19:30:22.762138 3328 raft_consensus.cc:729] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.762153 3329 ts_tablet_manager.cc:1397] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:22.762210 3328 consensus_queue.cc:260] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.762297 3329 raft_consensus.cc:348] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.762367 3329 raft_consensus.cc:374] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.762387 3329 raft_consensus.cc:729] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.762487 3328 ts_tablet_manager.cc:1428] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:22.762477 3329 consensus_queue.cc:260] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.762676 3329 ts_tablet_manager.cc:1428] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:22.763092 3332 tablet_bootstrap.cc:654] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.763715 3332 tablet_bootstrap.cc:492] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: No bootstrap required, opened a new log
I20260809 19:30:22.763758 3332 ts_tablet_manager.cc:1397] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:22.763888 3332 raft_consensus.cc:348] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.763944 3332 raft_consensus.cc:374] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.763957 3332 raft_consensus.cc:729] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.764019 3332 consensus_queue.cc:260] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.764181 3332 ts_tablet_manager.cc:1428] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:22.841995 3337 raft_consensus.cc:482] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:22.842079 3337 raft_consensus.cc:504] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.842489 3337 leader_election.cc:288] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541), e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099)
I20260809 19:30:22.845646 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.842748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:57821 (local address 127.31.57.194:43541)
0809 19:30:22.842890 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.842892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.842929 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:30:22.843019 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.843021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.843033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.843120 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:22.843322 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.843792 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.844744 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.845258 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.845339 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.845347 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.845534 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.845594 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260809 19:30:22.845657 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.842644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43541 (local address 127.31.57.193:57821)
0809 19:30:22.842868 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:22.842879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:22.842939 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:22.843128 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:22.843129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:22.843131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:22.843208 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.843210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.843862 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.843864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.844639 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.844642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.845343 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.845344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.845459 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.845462 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:22.845528 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:30:22.845568 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":42,"thread_start_us":134,"threads_started":1}
I20260809 19:30:22.845911 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e131e45f3e544153a45f9e92b0279827" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1" is_pre_election: true
I20260809 19:30:22.846005 2844 raft_consensus.cc:2454] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 0.
I20260809 19:30:22.846170 2493 leader_election.cc:302] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, 97e455b3b72a4cbbba4457a40bc104d1; no voters:
I20260809 19:30:22.846319 3337 raft_consensus.cc:2781] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:22.846372 3337 raft_consensus.cc:482] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:22.846387 3337 raft_consensus.cc:3037] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.846526 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.842654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.31.57.193:37791)
0809 19:30:22.842878 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:22.842889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:22.842955 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:22.843150 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:22.843151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:22.843153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:22.843231 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.843232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.843942 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.843942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.845428 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.845431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.846193 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.846194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.846268 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.846273 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:30:22.846323 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:30:22.846361 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":1,"thread_start_us":144,"threads_started":1}
I20260809 19:30:22.846566 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.842724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:37791 (local address 127.31.57.195:40099)
0809 19:30:22.842873 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.842875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.842938 (+ 63us) server_negotiation.cc:407] Connection header received
0809 19:30:22.843038 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.843039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.843047 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.843129 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:22.843338 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.843860 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.845563 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.846109 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.846182 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.846191 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.846416 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.846461 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260809 19:30:22.847180 3337 raft_consensus.cc:504] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.847332 3337 leader_election.cc:288] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 election: Requested vote from peers 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541), e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099)
I20260809 19:30:22.847472 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e131e45f3e544153a45f9e92b0279827" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1"
I20260809 19:30:22.847553 2844 raft_consensus.cc:3037] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.847584 3082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e131e45f3e544153a45f9e92b0279827" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" is_pre_election: true
I20260809 19:30:22.847584 3080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e131e45f3e544153a45f9e92b0279827" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
I20260809 19:30:22.847716 3082 raft_consensus.cc:2454] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 0.
I20260809 19:30:22.848214 2844 raft_consensus.cc:2454] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 1.
I20260809 19:30:22.848420 2493 leader_election.cc:302] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, 97e455b3b72a4cbbba4457a40bc104d1; no voters: e1ae66206fb14bfc820fd59ed6921d9e
I20260809 19:30:22.848526 3337 raft_consensus.cc:2781] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:22.848687 3337 raft_consensus.cc:686] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 1 LEADER]: Becoming Leader. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Running, Role: LEADER
I20260809 19:30:22.848780 3337 consensus_queue.cc:237] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.849469 2356 catalog_manager.cc:5495] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 560087126c064828bdf9c9c7399a7c32 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "560087126c064828bdf9c9c7399a7c32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:22.856544 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.854515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.31.57.193:34619)
0809 19:30:22.854647 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.854649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.854656 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:22.854693 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.854694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.854708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.854772 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:22.854872 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.855264 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.855928 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.856104 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.856154 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.856305 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:22.856392 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:22.856439 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.856473 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.856485 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260809 19:30:22.860903 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.858783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:52581 (local address 127.31.57.195:40099)
0809 19:30:22.858819 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.858821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.858905 (+ 84us) server_negotiation.cc:407] Connection header received
0809 19:30:22.858937 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.858938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.858944 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.858986 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:22.859106 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.859568 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.860230 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.860413 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.860495 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.860680 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:22.860713 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:22.860769 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.860826 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.860844 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:22.860935 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.858651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40099 (local address 127.31.57.193:52581)
0809 19:30:22.858843 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:22.858852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:22.858900 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:22.859025 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:22.859026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:22.859026 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:22.859070 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.859071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.859633 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.859633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.860161 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.860163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.860516 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.860517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.860579 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.860591 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:22.860771 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:22.860773 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:22.860822 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:22.860859 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260809 19:30:22.862844 2844 raft_consensus.cc:1261] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Refusing update from remote peer 560087126c064828bdf9c9c7399a7c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:22.862851 3080 raft_consensus.cc:3037] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.863075 3346 consensus_queue.cc:1035] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:22.863759 3080 raft_consensus.cc:1261] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Refusing update from remote peer 560087126c064828bdf9c9c7399a7c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:22.864048 3337 consensus_queue.cc:1035] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:22.870417 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.868412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:48803 (local address 127.31.57.193:34619)
0809 19:30:22.868480 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:22.868482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:22.868569 (+ 87us) server_negotiation.cc:407] Connection header received
0809 19:30:22.868615 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:22.868615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:22.868623 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:22.868667 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:22.868730 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.869116 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.869793 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.870047 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.870105 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.870228 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:22.870255 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:22.870297 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:22.870347 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:30:22.870365 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:30:22.870461 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:22.868377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:34619 (local address 127.31.57.195:48803)
0809 19:30:22.868526 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:22.868534 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:22.868578 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:22.868670 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:22.868670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:22.868671 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:22.868701 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.868702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.869169 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.869170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.869727 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:22.869730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:22.870105 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:22.870106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:22.870163 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:22.870167 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:22.870297 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:22.870299 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:22.870345 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:30:22.870384 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":117,"threads_started":1}
I20260809 19:30:22.876596 2355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:22.876698 2355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:22.877327 2355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Sending DeleteTablet for 3 replicas of tablet e131e45f3e544153a45f9e92b0279827
I20260809 19:30:22.877573 2811 tablet_service.cc:1514] Processing DeleteTablet for tablet e131e45f3e544153a45f9e92b0279827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:57890
I20260809 19:30:22.877592 2582 tablet_service.cc:1514] Processing DeleteTablet for tablet e131e45f3e544153a45f9e92b0279827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260809 19:30:22.877671 3311 tablet_replica.cc:330] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: stopping tablet replica
I20260809 19:30:22.877674 3315 tablet_replica.cc:330] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: stopping tablet replica
I20260809 19:30:22.877717 3311 raft_consensus.cc:2229] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:22.877729 3315 raft_consensus.cc:2229] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:22.877785 3315 raft_consensus.cc:2258] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.877815 3311 raft_consensus.cc:2258] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.877841 2355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:30:22.877951 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet e131e45f3e544153a45f9e92b0279827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:22 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53610
W20260809 19:30:22.877985 2355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:22.878029 3313 tablet_replica.cc:330] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: stopping tablet replica
I20260809 19:30:22.878068 3313 raft_consensus.cc:2229] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:22.878098 3313 raft_consensus.cc:2258] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:22.878264 3315 ts_tablet_manager.cc:1905] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.878543 3311 ts_tablet_manager.cc:1905] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.879698 3315 ts_tablet_manager.cc:1918] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:22.879753 3315 log.cc:1196] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/e131e45f3e544153a45f9e92b0279827
I20260809 19:30:22.879997 3315 ts_tablet_manager.cc:1939] T e131e45f3e544153a45f9e92b0279827 P 97e455b3b72a4cbbba4457a40bc104d1: Deleting consensus metadata
I20260809 19:30:22.880131 3311 ts_tablet_manager.cc:1918] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:22.880182 3311 log.cc:1196] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/e131e45f3e544153a45f9e92b0279827
I20260809 19:30:22.880299 2330 catalog_manager.cc:4841] TS 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541): tablet e131e45f3e544153a45f9e92b0279827 (table test-table [id=c9db476337fd4dd6b02466523dba14ef]) successfully deleted
I20260809 19:30:22.880388 3311 ts_tablet_manager.cc:1939] T e131e45f3e544153a45f9e92b0279827 P 560087126c064828bdf9c9c7399a7c32: Deleting consensus metadata
I20260809 19:30:22.880669 3043 tablet_service.cc:1467] Processing CreateTablet for tablet 26d29f6ac6a04701affccd8cd29f202b (DEFAULT_TABLE table=test-table [id=08213fbc9dac495cadc57784231cc5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.880676 2811 tablet_service.cc:1467] Processing CreateTablet for tablet 26d29f6ac6a04701affccd8cd29f202b (DEFAULT_TABLE table=test-table [id=08213fbc9dac495cadc57784231cc5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.880802 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d29f6ac6a04701affccd8cd29f202b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.880837 2335 catalog_manager.cc:4841] TS 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619): tablet e131e45f3e544153a45f9e92b0279827 (table test-table [id=c9db476337fd4dd6b02466523dba14ef]) successfully deleted
I20260809 19:30:22.880863 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d29f6ac6a04701affccd8cd29f202b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.881836 3332 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Bootstrap starting.
I20260809 19:30:22.882136 3328 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Bootstrap starting.
I20260809 19:30:22.882205 3332 tablet_bootstrap.cc:654] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.882201 2582 tablet_service.cc:1467] Processing CreateTablet for tablet 26d29f6ac6a04701affccd8cd29f202b (DEFAULT_TABLE table=test-table [id=08213fbc9dac495cadc57784231cc5e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:22.882308 2582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d29f6ac6a04701affccd8cd29f202b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:22.882483 3328 tablet_bootstrap.cc:654] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.882835 3332 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: No bootstrap required, opened a new log
I20260809 19:30:22.882890 3332 ts_tablet_manager.cc:1397] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:22.882997 3332 raft_consensus.cc:348] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.883085 3332 raft_consensus.cc:374] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.883113 3332 raft_consensus.cc:729] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e455b3b72a4cbbba4457a40bc104d1, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.883155 3332 consensus_queue.cc:260] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.883256 3332 ts_tablet_manager.cc:1428] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:22.883550 3329 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32: Bootstrap starting.
I20260809 19:30:22.883936 3329 tablet_bootstrap.cc:654] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:22.884426 3313 ts_tablet_manager.cc:1905] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:22.884620 3329 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32: No bootstrap required, opened a new log
I20260809 19:30:22.884672 3329 ts_tablet_manager.cc:1397] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:22.884857 3329 raft_consensus.cc:348] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.884928 3329 raft_consensus.cc:374] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.884949 3329 raft_consensus.cc:729] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.884989 3329 consensus_queue.cc:260] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.885088 3329 ts_tablet_manager.cc:1428] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:22.886006 3313 ts_tablet_manager.cc:1918] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:22.886068 3313 log.cc:1196] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/e131e45f3e544153a45f9e92b0279827
I20260809 19:30:22.886273 3313 ts_tablet_manager.cc:1939] T e131e45f3e544153a45f9e92b0279827 P e1ae66206fb14bfc820fd59ed6921d9e: Deleting consensus metadata
I20260809 19:30:22.886602 2332 catalog_manager.cc:4841] TS e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099): tablet e131e45f3e544153a45f9e92b0279827 (table test-table [id=c9db476337fd4dd6b02466523dba14ef]) successfully deleted
I20260809 19:30:22.895716 3328 tablet_bootstrap.cc:492] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: No bootstrap required, opened a new log
I20260809 19:30:22.895790 3328 ts_tablet_manager.cc:1397] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20260809 19:30:22.895931 3328 raft_consensus.cc:348] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.896006 3328 raft_consensus.cc:374] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:22.896039 3328 raft_consensus.cc:729] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ae66206fb14bfc820fd59ed6921d9e, State: Initialized, Role: FOLLOWER
I20260809 19:30:22.896091 3328 consensus_queue.cc:260] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.896204 3328 ts_tablet_manager.cc:1428] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:22.897426 3360 raft_consensus.cc:482] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:22.897492 3360 raft_consensus.cc:504] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.897637 3360 leader_election.cc:288] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541), e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099)
I20260809 19:30:22.897780 3080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d29f6ac6a04701affccd8cd29f202b" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" is_pre_election: true
I20260809 19:30:22.897802 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d29f6ac6a04701affccd8cd29f202b" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1" is_pre_election: true
I20260809 19:30:22.897848 3080 raft_consensus.cc:2454] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 0.
I20260809 19:30:22.897869 2844 raft_consensus.cc:2454] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 0.
I20260809 19:30:22.897991 2494 leader_election.cc:302] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:22.898126 3360 raft_consensus.cc:2781] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:22.898161 3360 raft_consensus.cc:482] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:22.898183 3360 raft_consensus.cc:3037] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.899060 3360 raft_consensus.cc:504] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.899211 3360 leader_election.cc:288] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 election: Requested vote from peers 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541), e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099)
I20260809 19:30:22.899405 3080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d29f6ac6a04701affccd8cd29f202b" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ae66206fb14bfc820fd59ed6921d9e"
I20260809 19:30:22.899330 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d29f6ac6a04701affccd8cd29f202b" candidate_uuid: "560087126c064828bdf9c9c7399a7c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e455b3b72a4cbbba4457a40bc104d1"
I20260809 19:30:22.899470 2844 raft_consensus.cc:3037] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.899470 3080 raft_consensus.cc:3037] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:22.900156 3080 raft_consensus.cc:2454] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 1.
I20260809 19:30:22.900159 2844 raft_consensus.cc:2454] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 560087126c064828bdf9c9c7399a7c32 in term 1.
I20260809 19:30:22.900313 2494 leader_election.cc:302] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560087126c064828bdf9c9c7399a7c32, e1ae66206fb14bfc820fd59ed6921d9e; no voters:
I20260809 19:30:22.900394 3360 raft_consensus.cc:2781] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:22.900431 3360 raft_consensus.cc:686] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [term 1 LEADER]: Becoming Leader. State: Replica: 560087126c064828bdf9c9c7399a7c32, State: Running, Role: LEADER
I20260809 19:30:22.900475 3360 consensus_queue.cc:237] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } }
I20260809 19:30:22.900980 2355 catalog_manager.cc:5495] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 560087126c064828bdf9c9c7399a7c32 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "560087126c064828bdf9c9c7399a7c32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560087126c064828bdf9c9c7399a7c32" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 34619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:22.907611 2844 raft_consensus.cc:1261] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Refusing update from remote peer 560087126c064828bdf9c9c7399a7c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:22.907682 3080 raft_consensus.cc:1261] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Refusing update from remote peer 560087126c064828bdf9c9c7399a7c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:22.907819 3337 consensus_queue.cc:1035] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e455b3b72a4cbbba4457a40bc104d1" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:22.907887 3360 consensus_queue.cc:1035] T 26d29f6ac6a04701affccd8cd29f202b P 560087126c064828bdf9c9c7399a7c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ae66206fb14bfc820fd59ed6921d9e" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:30:22.912072 3288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:30:24.431584 2355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:24.431672 2355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:24.432482 2355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fec4df7e43a64f64b0d94309b3b217a1: Sending DeleteTablet for 3 replicas of tablet 26d29f6ac6a04701affccd8cd29f202b
I20260809 19:30:24.432753 2811 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d29f6ac6a04701affccd8cd29f202b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:24 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:57890
I20260809 19:30:24.432806 2582 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d29f6ac6a04701affccd8cd29f202b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:24 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:46030
I20260809 19:30:24.432869 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2466
I20260809 19:30:24.432928 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d29f6ac6a04701affccd8cd29f202b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:24 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53610
I20260809 19:30:24.440528 3415 tablet_replica.cc:330] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: stopping tablet replica
I20260809 19:30:24.440613 3415 raft_consensus.cc:2229] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:24.440662 3415 raft_consensus.cc:2258] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:24.440780 3413 tablet_replica.cc:330] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: stopping tablet replica
I20260809 19:30:24.440840 3413 raft_consensus.cc:2229] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:24.440879 3413 raft_consensus.cc:2258] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:24.441082 3415 ts_tablet_manager.cc:1905] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:24.442553 3413 ts_tablet_manager.cc:1905] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:24.442616 3415 ts_tablet_manager.cc:1918] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:30:24.442723 3415 log.cc:1196] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/26d29f6ac6a04701affccd8cd29f202b
I20260809 19:30:24.443008 3415 ts_tablet_manager.cc:1939] T 26d29f6ac6a04701affccd8cd29f202b P e1ae66206fb14bfc820fd59ed6921d9e: Deleting consensus metadata
I20260809 19:30:24.443953 3413 ts_tablet_manager.cc:1918] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:30:24.444029 3413 log.cc:1196] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/26d29f6ac6a04701affccd8cd29f202b
I20260809 19:30:24.444228 3413 ts_tablet_manager.cc:1939] T 26d29f6ac6a04701affccd8cd29f202b P 97e455b3b72a4cbbba4457a40bc104d1: Deleting consensus metadata
I20260809 19:30:24.444555 2330 catalog_manager.cc:4841] TS 97e455b3b72a4cbbba4457a40bc104d1 (127.31.57.194:43541): tablet 26d29f6ac6a04701affccd8cd29f202b (table test-table [id=08213fbc9dac495cadc57784231cc5e5]) successfully deleted
I20260809 19:30:24.447046 2332 catalog_manager.cc:4841] TS e1ae66206fb14bfc820fd59ed6921d9e (127.31.57.195:40099): tablet 26d29f6ac6a04701affccd8cd29f202b (table test-table [id=08213fbc9dac495cadc57784231cc5e5]) successfully deleted
W20260809 19:30:24.447350 2335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:34619 (error 108)
I20260809 19:30:24.447956 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:24.447764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:34619 (local address 127.0.0.1:46058)
0809 19:30:24.447865 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:24.447916 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:34619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":68,"threads_started":1}
I20260809 19:30:24.447968 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2717
W20260809 19:30:24.448081 2335 catalog_manager.cc:4584] TS 560087126c064828bdf9c9c7399a7c32 (127.31.57.193:34619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26d29f6ac6a04701affccd8cd29f202b: Network error: Client connection negotiation failed: client connection to 127.31.57.193:34619: connect: Connection refused (error 111)
I20260809 19:30:24.456508 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2941
I20260809 19:30:24.463984 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2304
2026-08-09T19:30:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5739 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:24 dist-test-slave-x4qh krb5kdc[3424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:24 dist-test-slave-x4qh krb5kdc[3424](info): set up 2 sockets
Aug 09 19:30:24 dist-test-slave-x4qh krb5kdc[3424](info): commencing operation
krb5kdc: starting...
W20260809 19:30:25.627130 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:25 dist-test-slave-x4qh krb5kdc[3424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:25.734017 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40381
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:40381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:25.810726 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:25.810925 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:25.810947 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:25.812095 3445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:25.812135 3445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:25.812150 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:25.812184 3445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:25.812214 3445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:25.813665 3445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:40381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:25.813897 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:25.814113 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:25.818794 3453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:25.818915 3455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:25.818940 3445 server_base.cc:1034] running on GCE node
W20260809 19:30:25.818794 3452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:25.819470 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:25.819785 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:25.820955 3445 hybrid_clock.cc:648] HybridClock initialized: now 1786303825820934 us; error 34 us; skew 500 ppm
Aug 09 19:30:25 dist-test-slave-x4qh krb5kdc[3424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303825, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:25.823272 3445 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:25.823737 3445 webserver.cc:460] Webserver started at http://127.31.57.254:33485/ using document root <none> and password file <none>
I20260809 19:30:25.823969 3445 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:25.824040 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:25.824195 3445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:25.825248 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "b195bd21db674208beeef8647cc5e539"
format_stamp: "Formatted at 2026-08-09 19:30:25 on dist-test-slave-x4qh"
server_key: "56e881d17f670a9fd37b1f66babfef4c"
server_key_iv: "3b17ee93fd2324be528702c0c47d98fe"
server_key_version: "encryptionkey@0"
I20260809 19:30:25.825623 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "b195bd21db674208beeef8647cc5e539"
format_stamp: "Formatted at 2026-08-09 19:30:25 on dist-test-slave-x4qh"
server_key: "56e881d17f670a9fd37b1f66babfef4c"
server_key_iv: "3b17ee93fd2324be528702c0c47d98fe"
server_key_version: "encryptionkey@0"
I20260809 19:30:25.826844 3445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:30:25.827528 3467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:25.827695 3445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:25.827792 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "b195bd21db674208beeef8647cc5e539"
format_stamp: "Formatted at 2026-08-09 19:30:25 on dist-test-slave-x4qh"
server_key: "56e881d17f670a9fd37b1f66babfef4c"
server_key_iv: "3b17ee93fd2324be528702c0c47d98fe"
server_key_version: "encryptionkey@0"
I20260809 19:30:25.827879 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:25.854067 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:25.855819 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:25.855983 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:25.859946 3445 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:40381
I20260809 19:30:25.859994 3549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:40381 every 8 connection(s)
I20260809 19:30:25.860427 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:25.861315 3550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:25.863471 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539: Bootstrap starting.
I20260809 19:30:25.863991 3550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:25.864213 3550 log.cc:824] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:25.864863 3550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539: No bootstrap required, opened a new log
I20260809 19:30:25.866364 3550 raft_consensus.cc:348] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } }
I20260809 19:30:25.866493 3550 raft_consensus.cc:374] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:25.866515 3550 raft_consensus.cc:729] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b195bd21db674208beeef8647cc5e539, State: Initialized, Role: FOLLOWER
I20260809 19:30:25.866643 3550 consensus_queue.cc:260] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } }
I20260809 19:30:25.866725 3550 raft_consensus.cc:388] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:25.866748 3550 raft_consensus.cc:482] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:25.866776 3550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:25.867278 3550 raft_consensus.cc:504] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } }
I20260809 19:30:25.867415 3550 leader_election.cc:302] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b195bd21db674208beeef8647cc5e539; no voters:
I20260809 19:30:25.867551 3550 leader_election.cc:288] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:25.867601 3556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:25.867787 3550 sys_catalog.cc:564] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:25.867969 3556 raft_consensus.cc:686] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [term 1 LEADER]: Becoming Leader. State: Replica: b195bd21db674208beeef8647cc5e539, State: Running, Role: LEADER
I20260809 19:30:25.868062 3556 consensus_queue.cc:237] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } }
I20260809 19:30:25.868443 3558 sys_catalog.cc:455] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b195bd21db674208beeef8647cc5e539. Latest consensus state: current_term: 1 leader_uuid: "b195bd21db674208beeef8647cc5e539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } } }
I20260809 19:30:25.868453 3557 sys_catalog.cc:455] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b195bd21db674208beeef8647cc5e539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b195bd21db674208beeef8647cc5e539" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40381 } } }
I20260809 19:30:25.868556 3558 sys_catalog.cc:458] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:25.868561 3557 sys_catalog.cc:458] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:25.868602 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3445
I20260809 19:30:25.868680 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:25.868884 31975 external_mini_cluster.cc:1428] Setting key 7cc2abfb554d20b5f951354c9095c566
I20260809 19:30:25.869196 3566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:25.869736 3566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:25.871194 3566 catalog_manager.cc:1347] Generated new cluster ID: 5d7a531fc5eb43b2926bc4461c31140d
I20260809 19:30:25.871245 3566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:25 dist-test-slave-x4qh krb5kdc[3424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:25.879169 3566 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:25.879860 3566 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:25.880191 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:25.869490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.31.57.254:40381)
0809 19:30:25.869700 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:25.869708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:25.869740 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:30:25.869850 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:25.869857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:25.869908 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:25.870043 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:25.870816 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:25.872254 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:25.873331 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:25.874011 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:25.874113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:25.876795 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:25.876798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:25.876799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:25.876808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:25.878529 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:25.878919 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:25.878922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:25.878922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:25.879022 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:25.879204 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:25.879204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:25.879205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:25.879511 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:25.879670 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:25.879795 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 19:30:25.879951 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260809 19:30:25.886610 3566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b195bd21db674208beeef8647cc5e539: Generated new TSK 0
I20260809 19:30:25.886790 3566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:25.913969 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:25.992919 3593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:25.993098 3593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:25.993119 3593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:25.994184 3593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:25.994222 3593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:25.994271 3593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:25.995577 3593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:25.995795 3593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:25.996018 3593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:26.000839 3605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:26.001119 3614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:26.000869 3611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:26.001435 3593 server_base.cc:1034] running on GCE node
I20260809 19:30:26.001688 3593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:26.001946 3593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:26.003114 3593 hybrid_clock.cc:648] HybridClock initialized: now 1786303826003102 us; error 43 us; skew 500 ppm
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:26.005724 3593 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:26.006182 3593 webserver.cc:460] Webserver started at http://127.31.57.193:45173/ using document root <none> and password file <none>
I20260809 19:30:26.006438 3593 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:26.006506 3593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:26.006644 3593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:26.007700 3593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "4ae8dd0850e54c0087a631935c853b07"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "f5ea3a6b513d41dc1e2204ff8e268552"
server_key_iv: "174b6991231e342f730209f289865c27"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.008044 3593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "4ae8dd0850e54c0087a631935c853b07"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "f5ea3a6b513d41dc1e2204ff8e268552"
server_key_iv: "174b6991231e342f730209f289865c27"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.009338 3593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:26.010035 3624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.010206 3593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:26.010311 3593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "4ae8dd0850e54c0087a631935c853b07"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "f5ea3a6b513d41dc1e2204ff8e268552"
server_key_iv: "174b6991231e342f730209f289865c27"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.010391 3593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:26.018337 3593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:26.020148 3593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:26.020308 3593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:26.020527 3593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:26.020812 3593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:26.020866 3593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.020905 3593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:26.020932 3593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.026552 3593 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:40213
I20260809 19:30:26.026600 3813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:40213 every 8 connection(s)
I20260809 19:30:26.026863 3593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:26.028451 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3593
I20260809 19:30:26.028550 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:26.028761 31975 external_mini_cluster.cc:1428] Setting key dfc010417b176bf634082ed5a40caf78
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:26.037164 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.027489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:57467 (local address 127.31.57.254:40381)
0809 19:30:26.027565 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:26.027567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:26.027786 (+ 219us) server_negotiation.cc:407] Connection header received
0809 19:30:26.028196 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:26.028197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:26.028205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:26.028253 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:26.029502 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.030010 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.031506 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.031717 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.031790 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.034117 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:26.034119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:26.034119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:26.034125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:26.035653 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.036178 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.036179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.036180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.036233 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.036536 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.036536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.036537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.036643 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:26.036696 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:26.037025 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 19:30:26.037100 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:30:26.037408 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.027464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40381 (local address 127.31.57.193:57467)
0809 19:30:26.027665 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:26.027698 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:26.028118 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:26.028340 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:26.028343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:26.029161 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:26.029400 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.029407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.030098 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.030099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.031425 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.031429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.031820 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.031821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.031952 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.032474 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:26.032481 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:26.034020 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:26.035734 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.035742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.035745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.036063 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.036308 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.036309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.036310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.036463 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.036715 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:26.036723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:26.036880 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 19:30:26.037010 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 19:30:26.037178 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260809 19:30:26.037942 3814 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40381
I20260809 19:30:26.038054 3814 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:26.038229 3814 heartbeater.cc:511] Master 127.31.57.254:40381 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
I20260809 19:30:26.038746 3494 ts_manager.cc:194] Registered new tserver with Master: 4ae8dd0850e54c0087a631935c853b07 (127.31.57.193:40213)
I20260809 19:30:26.039484 3494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:57467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:26.054052 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:26.129472 3824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:26.129673 3824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:26.129695 3824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:26.130764 3824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:26.130837 3824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:26.130862 3824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:26.132182 3824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:26.132333 3824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:26.132496 3824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:26.136971 3835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:26.137045 3824 server_base.cc:1034] running on GCE node
W20260809 19:30:26.137022 3831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:26.136950 3832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:26.137307 3824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:26.137516 3824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:26.138692 3824 hybrid_clock.cc:648] HybridClock initialized: now 1786303826138675 us; error 33 us; skew 500 ppm
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:26.140942 3824 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:26.141358 3824 webserver.cc:460] Webserver started at http://127.31.57.194:35195/ using document root <none> and password file <none>
I20260809 19:30:26.141556 3824 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:26.141614 3824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:26.141745 3824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:26.142659 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "479e11ed6a5241aeba6a48a5f7f6680d"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "446789671bcd9da55456f52727d96b84"
server_key_iv: "042eccb782e42e16415d013e8df5cd1e"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.143002 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "479e11ed6a5241aeba6a48a5f7f6680d"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "446789671bcd9da55456f52727d96b84"
server_key_iv: "042eccb782e42e16415d013e8df5cd1e"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.144383 3824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:26.145004 3853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.145164 3824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:26.145251 3824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "479e11ed6a5241aeba6a48a5f7f6680d"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "446789671bcd9da55456f52727d96b84"
server_key_iv: "042eccb782e42e16415d013e8df5cd1e"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.145327 3824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:26.154356 3824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:26.156420 3824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:26.156554 3824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:26.156742 3824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:26.156999 3824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:26.157044 3824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.157071 3824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:26.157094 3824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.162262 3824 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:38359
I20260809 19:30:26.162307 4031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:38359 every 8 connection(s)
I20260809 19:30:26.162580 3824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:26.168247 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3824
I20260809 19:30:26.168370 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:26.168548 31975 external_mini_cluster.cc:1428] Setting key 6e4da34d31e7b78f7e7cdf0d0df341ae
I20260809 19:30:26.170723 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.163183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49137 (local address 127.31.57.254:40381)
0809 19:30:26.163229 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:26.163231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:26.163437 (+ 206us) server_negotiation.cc:407] Connection header received
0809 19:30:26.163820 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:26.163821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:26.163830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:26.163877 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:26.164622 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.165071 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.165876 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.166033 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.166077 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.167920 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:26.167921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:26.167921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:26.167927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:26.169400 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.169788 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.169789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.169790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.169856 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.170084 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.170085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.170086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.170191 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:26.170258 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:26.170537 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 19:30:26.170622 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:26.170992 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.163173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40381 (local address 127.31.57.194:49137)
0809 19:30:26.163314 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:26.163345 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:26.163732 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:26.163929 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:26.163931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:26.164398 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:26.164554 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.164558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.165143 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.165144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.165809 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.165810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.166097 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.166098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.166180 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.166603 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:26.166610 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:26.167829 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:26.169489 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.169496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.169498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.169700 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.169922 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.169922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.169923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.170022 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.170251 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:26.170258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:26.170401 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:30:26.170515 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:30:26.170656 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260809 19:30:26.171490 4033 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40381
I20260809 19:30:26.171559 4033 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:26.171754 4033 heartbeater.cc:511] Master 127.31.57.254:40381 requested a full tablet report, sending...
I20260809 19:30:26.172137 3494 ts_manager.cc:194] Registered new tserver with Master: 479e11ed6a5241aeba6a48a5f7f6680d (127.31.57.194:38359)
I20260809 19:30:26.172511 3494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:49137
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:26.191605 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:26.268095 4042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:26.268419 4042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:26.268456 4042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:26.269718 4042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:26.269757 4042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:26.269781 4042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:26.271055 4042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:26.271227 4042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:26.271404 4042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:26.275995 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:26.276073 4060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:26.276173 4042 server_base.cc:1034] running on GCE node
W20260809 19:30:26.276212 4049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:26.276399 4042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:26.276629 4042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:26.277786 4042 hybrid_clock.cc:648] HybridClock initialized: now 1786303826277761 us; error 33 us; skew 500 ppm
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:26.280249 4042 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:26.280589 4042 webserver.cc:460] Webserver started at http://127.31.57.195:41019/ using document root <none> and password file <none>
I20260809 19:30:26.280771 4042 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:26.280817 4042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:26.280898 4042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:26.281772 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "d8b1e171c21741a890dc25ee6cc9a6db"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "b5366cc665181e3757c89c56b02309b6"
server_key_iv: "6c04fa5048bee2ad38975cb8802328e8"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.282053 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "d8b1e171c21741a890dc25ee6cc9a6db"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "b5366cc665181e3757c89c56b02309b6"
server_key_iv: "6c04fa5048bee2ad38975cb8802328e8"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.283250 4042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:26.284032 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.284173 4042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:26.284250 4042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "d8b1e171c21741a890dc25ee6cc9a6db"
format_stamp: "Formatted at 2026-08-09 19:30:26 on dist-test-slave-x4qh"
server_key: "b5366cc665181e3757c89c56b02309b6"
server_key_iv: "6c04fa5048bee2ad38975cb8802328e8"
server_key_version: "encryptionkey@0"
I20260809 19:30:26.284298 4042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:26.289947 4042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:26.291958 4042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:26.292054 4042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:26.292202 4042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:26.292431 4042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:26.292462 4042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.292482 4042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:26.292497 4042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:26.297725 4042 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:40539
I20260809 19:30:26.297817 4236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:40539 every 8 connection(s)
I20260809 19:30:26.298089 4042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[3424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303826, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:26.305840 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4042
I20260809 19:30:26.305948 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:26.306135 31975 external_mini_cluster.cc:1428] Setting key 9f1c46ec4f32341d7de2b67c9a09239c
I20260809 19:30:26.306567 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.298803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:35385 (local address 127.31.57.254:40381)
0809 19:30:26.298844 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:26.298846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:26.299028 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:30:26.299498 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:26.299499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:26.299509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:26.299559 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:26.300363 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.300847 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.301757 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.301935 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.301983 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.303869 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:26.303870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:26.303871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:26.303878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:26.305157 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.305537 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.305538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.305538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.305594 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:26.305892 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:26.305893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:26.305894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:26.306028 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:26.306092 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:26.306396 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 19:30:26.306482 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:26.306711 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:26.298758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:40381 (local address 127.31.57.195:35385)
0809 19:30:26.298914 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:26.298945 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:26.299351 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:26.299606 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:26.299608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:26.300114 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:26.300291 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.300295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.300918 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.300919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.301686 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:26.301688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:26.302004 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:26.302005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:26.302099 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:26.302469 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:26.302477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:26.303770 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:26.305254 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.305260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.305262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.305462 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.305708 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:26.305709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:26.305709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:26.305822 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:26.306083 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:26.306089 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:26.306247 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 19:30:26.306359 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 19:30:26.306502 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260809 19:30:26.307090 4238 heartbeater.cc:347] Connected to a master server at 127.31.57.254:40381
I20260809 19:30:26.307173 4238 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:26.307303 4238 heartbeater.cc:511] Master 127.31.57.254:40381 requested a full tablet report, sending...
I20260809 19:30:26.307678 3494 ts_manager.cc:194] Registered new tserver with Master: d8b1e171c21741a890dc25ee6cc9a6db (127.31.57.195:40539)
I20260809 19:30:26.308020 3494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:35385
I20260809 19:30:26.317819 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:30:26.329344 3586 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:26.328698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.31.57.254:40381)
0809 19:30:26.328753 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:26.328755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:26.328811 (+ 56us) server_negotiation.cc:407] Connection header received
0809 19:30:26.328857 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:26.328858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:26.328864 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:26.328909 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:26.329303 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49170: BlockingRecv error: recv got EOF from 127.0.0.1:49170 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:30:26.330873 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3593
I20260809 19:30:26.340656 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3824
I20260809 19:30:26.347478 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4042
I20260809 19:30:26.357906 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3445
2026-08-09T19:30:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1890 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[4262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[4262](info): set up 2 sockets
Aug 09 19:30:26 dist-test-slave-x4qh krb5kdc[4262](info): commencing operation
krb5kdc: starting...
W20260809 19:30:27.510517 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:27.622141 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41687
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:27.697988 4284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:27.698179 4284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:27.698211 4284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:27.699182 4284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:27.699240 4284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:27.699262 4284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:27.699295 4284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:27.699319 4284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:27.700608 4284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:27.700862 4284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:27.701063 4284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:27.705626 4300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:27.705653 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:27.705680 4298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:27.705752 4284 server_base.cc:1034] running on GCE node
I20260809 19:30:27.706189 4284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:27.706453 4284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:27.707617 4284 hybrid_clock.cc:648] HybridClock initialized: now 1786303827707599 us; error 28 us; skew 500 ppm
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303827, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:27.709900 4284 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:27.710367 4284 webserver.cc:460] Webserver started at http://127.31.57.254:45761/ using document root <none> and password file <none>
I20260809 19:30:27.710613 4284 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:27.710676 4284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:27.710800 4284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:27.711880 4284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "cfb34d9649c344b1a439a95788f40954"
format_stamp: "Formatted at 2026-08-09 19:30:27 on dist-test-slave-x4qh"
server_key: "74a6027022d75b39216d0bfb723cee8d"
server_key_iv: "8463a1c8d8c5b854d21eb328f0b6f514"
server_key_version: "encryptionkey@0"
I20260809 19:30:27.712232 4284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "cfb34d9649c344b1a439a95788f40954"
format_stamp: "Formatted at 2026-08-09 19:30:27 on dist-test-slave-x4qh"
server_key: "74a6027022d75b39216d0bfb723cee8d"
server_key_iv: "8463a1c8d8c5b854d21eb328f0b6f514"
server_key_version: "encryptionkey@0"
I20260809 19:30:27.713544 4284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:27.714200 4311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:27.714370 4284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:27.714454 4284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "cfb34d9649c344b1a439a95788f40954"
format_stamp: "Formatted at 2026-08-09 19:30:27 on dist-test-slave-x4qh"
server_key: "74a6027022d75b39216d0bfb723cee8d"
server_key_iv: "8463a1c8d8c5b854d21eb328f0b6f514"
server_key_version: "encryptionkey@0"
I20260809 19:30:27.714534 4284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:27.731791 4284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:27.733418 4284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:27.733587 4284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:27.737699 4284 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41687
I20260809 19:30:27.737726 4388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41687 every 8 connection(s)
I20260809 19:30:27.738307 4284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:27.738976 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:27.741139 4389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954: Bootstrap starting.
I20260809 19:30:27.741706 4389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:27.742002 4389 log.cc:824] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:27.742650 4389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954: No bootstrap required, opened a new log
I20260809 19:30:27.744171 4389 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } }
I20260809 19:30:27.744297 4389 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:27.744369 4389 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb34d9649c344b1a439a95788f40954, State: Initialized, Role: FOLLOWER
I20260809 19:30:27.744506 4389 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } }
I20260809 19:30:27.744565 4389 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:27.744602 4389 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:27.744647 4389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:27.745466 4389 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } }
I20260809 19:30:27.745594 4389 leader_election.cc:302] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb34d9649c344b1a439a95788f40954; no voters:
I20260809 19:30:27.745783 4389 leader_election.cc:288] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:27.745831 4392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:27.746029 4392 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [term 1 LEADER]: Becoming Leader. State: Replica: cfb34d9649c344b1a439a95788f40954, State: Running, Role: LEADER
I20260809 19:30:27.746040 4389 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:27.746132 4392 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } }
I20260809 19:30:27.746454 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4284
I20260809 19:30:27.746483 4395 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb34d9649c344b1a439a95788f40954. Latest consensus state: current_term: 1 leader_uuid: "cfb34d9649c344b1a439a95788f40954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } } }
I20260809 19:30:27.746554 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:27.746555 4395 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:27.746712 4393 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfb34d9649c344b1a439a95788f40954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb34d9649c344b1a439a95788f40954" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41687 } } }
I20260809 19:30:27.746742 31975 external_mini_cluster.cc:1428] Setting key 5e8c285a08fd71130b4721d15816c4a7
I20260809 19:30:27.746762 4393 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:27.747177 4402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:27.747812 4402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:27.749286 4402 catalog_manager.cc:1347] Generated new cluster ID: c7ce5138fbc44ad8b21adbbf1b2bbb50
I20260809 19:30:27.749341 4402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:27.759779 4402 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:27.760245 4402 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:27.761898 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:27.747382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45126 (local address 127.31.57.254:41687)
0809 19:30:27.747611 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:27.747619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:27.747647 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:30:27.747746 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:27.747752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:27.747798 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:27.748596 (+ 798us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:27.748679 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:27.749947 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:27.750937 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:27.751517 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:27.751598 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:27.754196 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:27.754200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:27.754200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:27.754212 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:27.760277 (+ 6065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:27.760687 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:27.760690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:27.760691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:27.760807 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:27.761016 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:27.761017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:27.761017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:27.761280 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:27.761411 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:27.761522 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 19:30:27.761670 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260809 19:30:27.762836 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:27.766172 4402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb34d9649c344b1a439a95788f40954: Generated new TSK 0
I20260809 19:30:27.766340 4402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:30:27.779441 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:27.776279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.31.57.254:41687)
0809 19:30:27.776353 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:27.776354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:27.776395 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:30:27.776444 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:27.776445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:27.776453 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:27.776512 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:27.776806 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:27.777493 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:27.778098 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:27.778389 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:27.778454 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:27.779007 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:27.779009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:27.779009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:27.779192 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:27.779208 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:27.779214 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:27.779271 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:27.779317 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 19:30:27.779340 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 19:30:27.781934 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4284
2026-08-09T19:30:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1426 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4439](info): set up 2 sockets
Aug 09 19:30:27 dist-test-slave-x4qh krb5kdc[4439](info): commencing operation
krb5kdc: starting...
W20260809 19:30:28.944553 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:29.051994 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:38135
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:38135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:29.125852 4454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:29.126029 4454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:29.126046 4454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:29.127068 4454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:29.127104 4454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:29.127116 4454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:29.127127 4454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:29.127178 4454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:29.128482 4454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:38135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:38135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:29.128715 4454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:29.128963 4454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:29.133507 4463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.133667 4454 server_base.cc:1034] running on GCE node
W20260809 19:30:29.133625 4460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:29.133524 4466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.134055 4454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:29.134347 4454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:29.135432 4454 hybrid_clock.cc:648] HybridClock initialized: now 1786303829135422 us; error 28 us; skew 500 ppm
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:29.137702 4454 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:29.138154 4454 webserver.cc:460] Webserver started at http://127.31.57.254:38787/ using document root <none> and password file <none>
I20260809 19:30:29.138381 4454 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:29.138453 4454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:29.138628 4454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:29.139654 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "a0a8969a60ca47339cc77876c20b6248"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "a2b09c3037f9a356d9ef5b38de56f429"
server_key_iv: "2e719947f75403ffc99f86d425c57e9f"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.140014 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "a0a8969a60ca47339cc77876c20b6248"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "a2b09c3037f9a356d9ef5b38de56f429"
server_key_iv: "2e719947f75403ffc99f86d425c57e9f"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.141260 4454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:29.141930 4477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.142102 4454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:29.142200 4454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "a0a8969a60ca47339cc77876c20b6248"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "a2b09c3037f9a356d9ef5b38de56f429"
server_key_iv: "2e719947f75403ffc99f86d425c57e9f"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.142268 4454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:29.163908 4454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:29.165578 4454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:29.165735 4454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:29.169626 4454 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:38135
I20260809 19:30:29.169664 4565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:38135 every 8 connection(s)
I20260809 19:30:29.169996 4454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:29.170917 4566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:29.172967 4566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248: Bootstrap starting.
I20260809 19:30:29.173473 4566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:29.173698 4566 log.cc:824] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:29.174314 4566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248: No bootstrap required, opened a new log
I20260809 19:30:29.175761 4566 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } }
I20260809 19:30:29.175887 4566 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:29.175907 4566 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a8969a60ca47339cc77876c20b6248, State: Initialized, Role: FOLLOWER
I20260809 19:30:29.175997 4566 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } }
I20260809 19:30:29.176039 4566 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:29.176057 4566 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:29.176081 4566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:29.176870 4566 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } }
I20260809 19:30:29.176975 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4454
I20260809 19:30:29.176990 4566 leader_election.cc:302] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0a8969a60ca47339cc77876c20b6248; no voters:
I20260809 19:30:29.177076 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:29.177143 4566 leader_election.cc:288] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:29.177189 4569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:29.177296 31975 external_mini_cluster.cc:1428] Setting key 889ab61a1dd3897cf3c57112f47cde03
I20260809 19:30:29.177357 4569 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [term 1 LEADER]: Becoming Leader. State: Replica: a0a8969a60ca47339cc77876c20b6248, State: Running, Role: LEADER
I20260809 19:30:29.177443 4566 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:29.177428 4569 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } }
I20260809 19:30:29.177874 4571 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0a8969a60ca47339cc77876c20b6248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } } }
I20260809 19:30:29.178036 4571 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:29.177913 4572 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0a8969a60ca47339cc77876c20b6248. Latest consensus state: current_term: 1 leader_uuid: "a0a8969a60ca47339cc77876c20b6248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a8969a60ca47339cc77876c20b6248" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 38135 } } }
I20260809 19:30:29.178282 4572 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:29.178542 4585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:29.179083 4585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:29.180965 4585 catalog_manager.cc:1347] Generated new cluster ID: b0a88294bb794c0ba2346722443e5839
I20260809 19:30:29.181030 4585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:29.188970 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.177768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34026 (local address 127.31.57.254:38135)
0809 19:30:29.178087 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:29.178097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:29.178133 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:30:29.178252 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:29.178259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:29.178323 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:29.179470 (+ 1147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:29.179587 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.180725 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.181808 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.182462 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.182576 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.185392 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:29.185395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:29.185396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:29.185406 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:29.187323 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.187744 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.187746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.187747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.187859 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.188079 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.188080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.188080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.188360 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:29.188505 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:29.188646 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 19:30:29.188774 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":66,"threads_started":1}
I20260809 19:30:29.189463 4585 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:29.189977 4585 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:29.192979 4585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0a8969a60ca47339cc77876c20b6248: Generated new TSK 0
I20260809 19:30:29.193146 4585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:29.219594 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:29.294157 4603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:29.294328 4603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:29.294349 4603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:29.295415 4603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:29.295466 4603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:29.295512 4603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:29.296838 4603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:29.297003 4603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:29.297227 4603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:29.301950 4610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:29.301995 4613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.301965 4603 server_base.cc:1034] running on GCE node
W20260809 19:30:29.301950 4611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.302343 4603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:29.302608 4603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:29.303776 4603 hybrid_clock.cc:648] HybridClock initialized: now 1786303829303769 us; error 41 us; skew 500 ppm
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:29.306200 4603 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:29.306595 4603 webserver.cc:460] Webserver started at http://127.31.57.193:35023/ using document root <none> and password file <none>
I20260809 19:30:29.306833 4603 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:29.306900 4603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:29.307047 4603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:29.308036 4603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "e278a72d96f74667814a65003cfd05e9"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "742d19ea47771c90d2da20772f667436"
server_key_iv: "de19fe22773ebe40362bce52618055ba"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.308351 4603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "e278a72d96f74667814a65003cfd05e9"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "742d19ea47771c90d2da20772f667436"
server_key_iv: "de19fe22773ebe40362bce52618055ba"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.309478 4603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:29.310124 4621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.310289 4603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:29.310381 4603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "e278a72d96f74667814a65003cfd05e9"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "742d19ea47771c90d2da20772f667436"
server_key_iv: "de19fe22773ebe40362bce52618055ba"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.310465 4603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:29.331115 4603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:29.332664 4603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:29.332839 4603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:29.333062 4603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:29.333361 4603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:29.333418 4603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.333449 4603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:29.333477 4603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.338973 4603 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:40899
I20260809 19:30:29.339058 4804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:40899 every 8 connection(s)
I20260809 19:30:29.339308 4603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:29.343891 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4603
I20260809 19:30:29.343987 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:29.344182 31975 external_mini_cluster.cc:1428] Setting key 5e0733c06d5d36baf8f00a5d054c5e1c
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:29.348692 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.340001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:38135 (local address 127.31.57.193:36663)
0809 19:30:29.340160 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:29.340191 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:29.340601 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:29.340793 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:29.340796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:29.341276 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:29.341471 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.341476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.342120 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.342120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.342996 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.342998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.343364 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.343365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.343532 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.343949 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:29.343959 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:29.345578 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:29.347436 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.347442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.347443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.347643 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.347865 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.347866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.347866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.347955 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.348252 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:29.348256 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:29.348370 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 19:30:29.348447 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:30:29.348562 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260809 19:30:29.348732 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.340020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:36663 (local address 127.31.57.254:38135)
0809 19:30:29.340066 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:29.340070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:29.340265 (+ 195us) server_negotiation.cc:407] Connection header received
0809 19:30:29.340686 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:29.340687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:29.340697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:29.340745 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:29.341558 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.342042 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.343101 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.343296 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.343345 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.345673 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:29.345674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:29.345675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:29.345682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:29.347280 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.347718 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.347720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.347720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.347792 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.348021 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.348022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.348022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.348183 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:29.348259 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:29.348508 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 19:30:29.348596 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:29.349154 4805 heartbeater.cc:347] Connected to a master server at 127.31.57.254:38135
I20260809 19:30:29.349247 4805 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:29.349442 4805 heartbeater.cc:511] Master 127.31.57.254:38135 requested a full tablet report, sending...
I20260809 19:30:29.349925 4512 ts_manager.cc:194] Registered new tserver with Master: e278a72d96f74667814a65003cfd05e9 (127.31.57.193:40899)
I20260809 19:30:29.350471 4512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:36663
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:29.369550 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:29.442052 4812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:29.442256 4812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:29.442293 4812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:29.443480 4812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:29.443543 4812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:29.443590 4812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:29.444960 4812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:29.445207 4812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:29.445420 4812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:29.449970 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.449992 4812 server_base.cc:1034] running on GCE node
W20260809 19:30:29.450078 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:29.450158 4823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.450392 4812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:29.450631 4812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:29.451778 4812 hybrid_clock.cc:648] HybridClock initialized: now 1786303829451767 us; error 33 us; skew 500 ppm
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:29.454082 4812 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:29.454429 4812 webserver.cc:460] Webserver started at http://127.31.57.194:39581/ using document root <none> and password file <none>
I20260809 19:30:29.454658 4812 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:29.454774 4812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:29.454921 4812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:29.455942 4812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "eb1ae5de07504f30a71aa10e24a81b06"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "9cf62e015777d5e026ce0f8233e9f9b9"
server_key_iv: "65cc525b9fe769a1c79d9165cab02789"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.456315 4812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "eb1ae5de07504f30a71aa10e24a81b06"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "9cf62e015777d5e026ce0f8233e9f9b9"
server_key_iv: "65cc525b9fe769a1c79d9165cab02789"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.457445 4812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:29.458032 4832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.458184 4812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:29.458267 4812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "eb1ae5de07504f30a71aa10e24a81b06"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "9cf62e015777d5e026ce0f8233e9f9b9"
server_key_iv: "65cc525b9fe769a1c79d9165cab02789"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.458339 4812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:29.471236 4812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:29.472924 4812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:29.473075 4812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:29.473282 4812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:29.473572 4812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:29.473604 4812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.473623 4812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:29.473636 4812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.479122 4812 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:40993
I20260809 19:30:29.479207 5002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:40993 every 8 connection(s)
I20260809 19:30:29.479504 4812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:29.483587 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4812
I20260809 19:30:29.483686 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:29.483879 31975 external_mini_cluster.cc:1428] Setting key b6dc042b7d5dffca0ce425a819c3d393
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:29.487936 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.480126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:38135 (local address 127.31.57.194:49461)
0809 19:30:29.480264 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:29.480298 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:29.480673 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:29.480857 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:29.480860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:29.481323 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:29.481477 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.481481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.482055 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.482056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.482771 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.482772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.483094 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.483095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.483189 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.483601 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:29.483611 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:29.485174 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:29.486685 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.486692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.486694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.486903 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.487112 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.487112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.487113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.487216 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.487513 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:29.487518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:29.487615 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 19:30:29.487696 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:30:29.487806 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260809 19:30:29.487950 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.480163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49461 (local address 127.31.57.254:38135)
0809 19:30:29.480221 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:29.480222 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:29.480372 (+ 150us) server_negotiation.cc:407] Connection header received
0809 19:30:29.480745 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:29.480746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:29.480755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:29.480815 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:29.481545 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.481983 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.482852 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.483026 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.483075 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.485285 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:29.485287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:29.485287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:29.485294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:29.486587 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.486980 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.486982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.486982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.487041 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.487287 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.487288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.487288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.487444 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:29.487511 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:29.487741 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 19:30:29.487826 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:30:29.488556 5003 heartbeater.cc:347] Connected to a master server at 127.31.57.254:38135
I20260809 19:30:29.488651 5003 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:29.488827 5003 heartbeater.cc:511] Master 127.31.57.254:38135 requested a full tablet report, sending...
I20260809 19:30:29.489161 4512 ts_manager.cc:194] Registered new tserver with Master: eb1ae5de07504f30a71aa10e24a81b06 (127.31.57.194:40993)
I20260809 19:30:29.489503 4512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:49461
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:29.507910 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:29.584952 5009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:29.585121 5009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:29.585142 5009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:29.586171 5009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:29.586210 5009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:29.586234 5009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:29.587641 5009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:38135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:29.587867 5009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:29.588070 5009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:29.592638 5020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:29.592676 5018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:29.592690 5017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:29.593003 5009 server_base.cc:1034] running on GCE node
I20260809 19:30:29.593205 5009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:29.593442 5009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:29.594576 5009 hybrid_clock.cc:648] HybridClock initialized: now 1786303829594568 us; error 39 us; skew 500 ppm
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:29.596886 5009 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:29.597287 5009 webserver.cc:460] Webserver started at http://127.31.57.195:35133/ using document root <none> and password file <none>
I20260809 19:30:29.597512 5009 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:29.597565 5009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:29.597711 5009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:29.598619 5009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "b7bbd98e0df04ee989cfd644c1f0a9c3"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "fa4e67bdf5f62d73bc53d4f424ee102c"
server_key_iv: "6b6fcda766a74142799faea1de08de80"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.598939 5009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "b7bbd98e0df04ee989cfd644c1f0a9c3"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "fa4e67bdf5f62d73bc53d4f424ee102c"
server_key_iv: "6b6fcda766a74142799faea1de08de80"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.600217 5009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:29.600894 5033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.601056 5009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:29.601137 5009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "b7bbd98e0df04ee989cfd644c1f0a9c3"
format_stamp: "Formatted at 2026-08-09 19:30:29 on dist-test-slave-x4qh"
server_key: "fa4e67bdf5f62d73bc53d4f424ee102c"
server_key_iv: "6b6fcda766a74142799faea1de08de80"
server_key_version: "encryptionkey@0"
I20260809 19:30:29.601200 5009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:29.618130 5009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:29.620127 5009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:29.620232 5009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:29.620373 5009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:29.620654 5009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:29.620685 5009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.620704 5009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:29.620750 5009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:29.626245 5009 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:43415
I20260809 19:30:29.626298 5210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:43415 every 8 connection(s)
I20260809 19:30:29.626542 5009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:29.633065 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 5009
I20260809 19:30:29.633162 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:29.633330 31975 external_mini_cluster.cc:1428] Setting key d0644d97dfdc07599679fede0ec43a06
I20260809 19:30:29.634938 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.627104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:39003 (local address 127.31.57.254:38135)
0809 19:30:29.627186 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:29.627188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:29.627384 (+ 196us) server_negotiation.cc:407] Connection header received
0809 19:30:29.627805 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:29.627806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:29.627816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:29.627860 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:29.628626 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.629104 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.629978 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.630144 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.630189 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.632128 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:29.632130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:29.632130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:29.632137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:29.633703 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.634194 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.634195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.634196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.634278 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.634480 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.634481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.634482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.634576 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:29.634630 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:29.634811 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 19:30:29.634884 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 19:30:29.635066 5215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.627065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:38135 (local address 127.31.57.195:39003)
0809 19:30:29.627225 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:29.627254 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:29.627720 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:29.627914 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:29.627917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:29.628395 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:29.628555 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.628559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.629180 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.629180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.629903 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:29.629905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.630210 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.630210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:29.630314 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.630670 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:29.630677 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:29.632032 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:29.633816 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.633822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.633824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.634122 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.634337 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:29.634337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:29.634338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:29.634431 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:29.634622 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:29.634624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:29.634712 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 19:30:29.634778 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:30:29.634895 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260809 19:30:29.635450 5211 heartbeater.cc:347] Connected to a master server at 127.31.57.254:38135
I20260809 19:30:29.635535 5211 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:29.635711 5211 heartbeater.cc:511] Master 127.31.57.254:38135 requested a full tablet report, sending...
I20260809 19:30:29.636008 4512 ts_manager.cc:194] Registered new tserver with Master: b7bbd98e0df04ee989cfd644c1f0a9c3 (127.31.57.195:43415)
I20260809 19:30:29.636361 4512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:39003
I20260809 19:30:29.645084 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[4439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303829, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:29.667104 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:29.660033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34042 (local address 127.31.57.254:38135)
0809 19:30:29.660086 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:29.660088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:29.660188 (+ 100us) server_negotiation.cc:407] Connection header received
0809 19:30:29.660250 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:29.660251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:29.660260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:29.660309 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:29.660876 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.661310 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.662080 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:29.662267 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:29.662341 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:29.664820 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:29.664822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:29.664822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:29.664829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:29.666134 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.666453 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.666455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.666455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.666542 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:29.666711 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:29.666712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:29.666712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:29.666848 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:29.666912 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:29.667000 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 19:30:29.667051 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 19:30:29.667426 4512 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34042
I20260809 19:30:29.667802 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4603
I20260809 19:30:29.680650 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4812
I20260809 19:30:29.703876 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 5009
I20260809 19:30:29.712949 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4454
2026-08-09T19:30:29Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1928 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[5231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[5231](info): set up 2 sockets
Aug 09 19:30:29 dist-test-slave-x4qh krb5kdc[5231](info): commencing operation
krb5kdc: starting...
W20260809 19:30:30.852154 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:30 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:30.959829 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41717
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:31.038120 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:31.038321 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:31.038353 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:31.039505 5246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:31.039564 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:31.039587 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:31.039609 5246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:31.039636 5246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:31.041054 5246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:31.041302 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:31.041510 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:31.046145 5257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.046159 5260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.046159 5256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:31.046418 5246 server_base.cc:1034] running on GCE node
I20260809 19:30:31.046655 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:31.046937 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:31.048085 5246 hybrid_clock.cc:648] HybridClock initialized: now 1786303831048067 us; error 30 us; skew 500 ppm
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:31.050487 5246 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:31.050963 5246 webserver.cc:460] Webserver started at http://127.31.57.254:39393/ using document root <none> and password file <none>
I20260809 19:30:31.051213 5246 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:31.051258 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:31.051468 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:31.052510 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "cab28d66f1794e699f76ca950f86488a"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "58ab81d370bce5d4e181bb2033d04fff"
server_key_iv: "5ac0c1152b32db5654da3e4e0718d9c2"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.052868 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "cab28d66f1794e699f76ca950f86488a"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "58ab81d370bce5d4e181bb2033d04fff"
server_key_iv: "5ac0c1152b32db5654da3e4e0718d9c2"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.054193 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:31.054867 5272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.055037 5246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:31.055138 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "cab28d66f1794e699f76ca950f86488a"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "58ab81d370bce5d4e181bb2033d04fff"
server_key_iv: "5ac0c1152b32db5654da3e4e0718d9c2"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.055208 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:31.068081 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:31.069723 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:31.069888 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:31.074163 5246 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41717
I20260809 19:30:31.074195 5356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41717 every 8 connection(s)
I20260809 19:30:31.074668 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:31.075313 5358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:31.077332 5358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a: Bootstrap starting.
I20260809 19:30:31.077836 5358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:31.078100 5358 log.cc:824] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:31.078787 5358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a: No bootstrap required, opened a new log
I20260809 19:30:31.080423 5358 raft_consensus.cc:348] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } }
I20260809 19:30:31.080552 5358 raft_consensus.cc:374] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:31.080574 5358 raft_consensus.cc:729] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab28d66f1794e699f76ca950f86488a, State: Initialized, Role: FOLLOWER
I20260809 19:30:31.080701 5358 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } }
I20260809 19:30:31.080782 5358 raft_consensus.cc:388] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:31.080809 5358 raft_consensus.cc:482] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:31.080837 5358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:31.081653 5358 raft_consensus.cc:504] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } }
I20260809 19:30:31.081769 5358 leader_election.cc:302] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cab28d66f1794e699f76ca950f86488a; no voters:
I20260809 19:30:31.081885 5358 leader_election.cc:288] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:31.081949 5365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:31.082113 5365 raft_consensus.cc:686] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [term 1 LEADER]: Becoming Leader. State: Replica: cab28d66f1794e699f76ca950f86488a, State: Running, Role: LEADER
I20260809 19:30:31.082151 5358 sys_catalog.cc:564] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:31.082217 5365 consensus_queue.cc:237] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } }
I20260809 19:30:31.082609 5366 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cab28d66f1794e699f76ca950f86488a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } } }
I20260809 19:30:31.082639 5368 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cab28d66f1794e699f76ca950f86488a. Latest consensus state: current_term: 1 leader_uuid: "cab28d66f1794e699f76ca950f86488a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab28d66f1794e699f76ca950f86488a" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41717 } } }
I20260809 19:30:31.082805 5368 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:31.082754 5366 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:31.083276 5380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:31.083881 5380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:31.084142 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 5246
I20260809 19:30:31.084233 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:31.084425 31975 external_mini_cluster.cc:1428] Setting key 7281abf95a96cffecbab910a19fa65d5
I20260809 19:30:31.085605 5380 catalog_manager.cc:1347] Generated new cluster ID: 1a05ce0e4d734a99bc21cb8952336511
I20260809 19:30:31.085656 5380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:31.093793 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.084906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32782 (local address 127.31.57.254:41717)
0809 19:30:31.085054 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.085065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.085168 (+ 103us) server_negotiation.cc:407] Connection header received
0809 19:30:31.085326 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.085332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.085373 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.085536 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.086167 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.087114 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.087903 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.088325 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.088440 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.090570 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.090573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.090574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.090584 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.092198 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.092585 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.092588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.092588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.092681 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.092915 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.092915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.092916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.093185 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.093371 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.093440 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.093584 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260809 19:30:31.100476 5380 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:31.100914 5380 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:31.109422 5380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cab28d66f1794e699f76ca950f86488a: Generated new TSK 0
I20260809 19:30:31.109606 5380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:31.143579 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:31.228260 5400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:31.228474 5400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:31.228511 5400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:31.229566 5400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:31.229633 5400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:31.229673 5400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:31.231071 5400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:31.231316 5400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:31.231550 5400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:31.236289 5408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.236279 5412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.236279 5407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:31.236541 5400 server_base.cc:1034] running on GCE node
I20260809 19:30:31.236778 5400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:31.237041 5400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:31.238196 5400 hybrid_clock.cc:648] HybridClock initialized: now 1786303831238192 us; error 35 us; skew 500 ppm
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:31.240614 5400 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:31.241024 5400 webserver.cc:460] Webserver started at http://127.31.57.193:41607/ using document root <none> and password file <none>
I20260809 19:30:31.241257 5400 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:31.241325 5400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:31.241472 5400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:31.242435 5400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "eb58b75061c44887adf8d1a13f0eb76b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "a18bc299d4252f9690b71dc4ea904d05"
server_key_iv: "47fcdf3e7dce1018141b88119dc53614"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.242771 5400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "eb58b75061c44887adf8d1a13f0eb76b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "a18bc299d4252f9690b71dc4ea904d05"
server_key_iv: "47fcdf3e7dce1018141b88119dc53614"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.244243 5400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:31.244974 5419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.245149 5400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:31.245245 5400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "eb58b75061c44887adf8d1a13f0eb76b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "a18bc299d4252f9690b71dc4ea904d05"
server_key_iv: "47fcdf3e7dce1018141b88119dc53614"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.245324 5400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:31.261056 5400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:31.262705 5400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:31.262900 5400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:31.263114 5400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:31.263470 5400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:31.263533 5400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.263573 5400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:31.263602 5400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.269349 5400 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:37727
I20260809 19:30:31.269423 5589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:37727 every 8 connection(s)
I20260809 19:30:31.269718 5400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:31.278503 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 5400
I20260809 19:30:31.278594 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:31.278810 31975 external_mini_cluster.cc:1428] Setting key 8ba1e8b3fe0f05bcba9d37eec0ba672f
I20260809 19:30:31.279287 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.270444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:52583 (local address 127.31.57.254:41717)
0809 19:30:31.270502 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.270504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.270730 (+ 226us) server_negotiation.cc:407] Connection header received
0809 19:30:31.271656 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.271657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.271672 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.271725 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.272311 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.272799 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.273812 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.274035 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.274130 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.276205 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.276206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.276207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.276215 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.277674 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.278122 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.278123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.278123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.278182 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.278511 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.278513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.278513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.278698 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.278762 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.279087 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.279202 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:31.279457 5592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.270403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41717 (local address 127.31.57.193:52583)
0809 19:30:31.270613 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:31.270642 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:31.271539 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:31.271784 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:31.271787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:31.272038 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:31.272205 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.272209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.272916 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.272917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.273710 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.273712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.274151 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.274152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.274271 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.274744 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:31.274752 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:31.276093 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:31.277796 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.277805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.277807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.278026 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.278262 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.278262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.278263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.278424 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.278773 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:31.278781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:31.278933 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 19:30:31.279047 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:30:31.279225 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260809 19:30:31.279966 5590 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41717
I20260809 19:30:31.280071 5590 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:31.280274 5590 heartbeater.cc:511] Master 127.31.57.254:41717 requested a full tablet report, sending...
I20260809 19:30:31.280719 5306 ts_manager.cc:194] Registered new tserver with Master: eb58b75061c44887adf8d1a13f0eb76b (127.31.57.193:37727)
I20260809 19:30:31.281383 5306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:52583
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:31.302875 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:31.379719 5596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:31.379892 5596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:31.379915 5596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:31.380908 5596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:31.380937 5596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:31.380961 5596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:31.382296 5596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:31.382462 5596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:31.382602 5596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:31.387308 5610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.387310 5608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.387339 5607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:31.387509 5596 server_base.cc:1034] running on GCE node
I20260809 19:30:31.387722 5596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:31.387959 5596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:31.389119 5596 hybrid_clock.cc:648] HybridClock initialized: now 1786303831389114 us; error 33 us; skew 500 ppm
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:31.391326 5596 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:31.391722 5596 webserver.cc:460] Webserver started at http://127.31.57.194:39659/ using document root <none> and password file <none>
I20260809 19:30:31.391945 5596 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:31.392037 5596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:31.392170 5596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:31.393038 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "87a5137a2d3743899209016dc10fc26b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "eed2eea0ea18a4e07e9c9c1b89b9b43d"
server_key_iv: "8b3bd8f037ff3b7c198fc7d8c88acc77"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.393323 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "87a5137a2d3743899209016dc10fc26b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "eed2eea0ea18a4e07e9c9c1b89b9b43d"
server_key_iv: "8b3bd8f037ff3b7c198fc7d8c88acc77"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.394486 5596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:31.395074 5621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.395252 5596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.395316 5596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "87a5137a2d3743899209016dc10fc26b"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "eed2eea0ea18a4e07e9c9c1b89b9b43d"
server_key_iv: "8b3bd8f037ff3b7c198fc7d8c88acc77"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.395360 5596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:31.414700 5596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:31.416723 5596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:31.416827 5596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:31.416970 5596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:31.417246 5596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:31.417279 5596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.417300 5596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:31.417343 5596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.422710 5596 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:34999
I20260809 19:30:31.422760 5795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:34999 every 8 connection(s)
I20260809 19:30:31.423046 5596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:31.427327 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 5596
I20260809 19:30:31.427461 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:31.427636 31975 external_mini_cluster.cc:1428] Setting key c4f8c48ac0328eca54b6b631a3939e17
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:31.431550 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.423685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49503 (local address 127.31.57.254:41717)
0809 19:30:31.423730 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.423732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.423914 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:30:31.424746 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.424747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.424757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.424816 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.425348 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.425811 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.426648 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.426815 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.426863 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.429007 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.429008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.429009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.429016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.430186 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.430568 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.430570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.430570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.430629 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.430892 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.430893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.430893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.431000 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.431067 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.431347 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.431469 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:31.431712 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.423626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41717 (local address 127.31.57.194:49503)
0809 19:30:31.423790 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:31.423827 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:31.424660 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:31.424892 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:31.424895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:31.425104 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:31.425266 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.425270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.425882 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.425883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.426577 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.426578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.426884 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.426885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.426981 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.427431 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:31.427440 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:31.428912 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:31.430283 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.430290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.430292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.430492 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.430702 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.430702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.430703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.430815 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.431072 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:31.431080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:31.431218 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 19:30:31.431326 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:30:31.431495 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260809 19:30:31.432129 5797 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41717
I20260809 19:30:31.432212 5797 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:31.432387 5797 heartbeater.cc:511] Master 127.31.57.254:41717 requested a full tablet report, sending...
I20260809 19:30:31.432752 5306 ts_manager.cc:194] Registered new tserver with Master: 87a5137a2d3743899209016dc10fc26b (127.31.57.194:34999)
I20260809 19:30:31.433149 5306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:49503
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:31.451536 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:31.528585 5806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:31.528760 5806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:31.528782 5806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:31.529853 5806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:31.529891 5806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:31.529940 5806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:31.531360 5806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:31.531641 5806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:31.531857 5806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:31.536473 5819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.536466 5821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:31.536489 5818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:31.536696 5806 server_base.cc:1034] running on GCE node
I20260809 19:30:31.536890 5806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:31.537129 5806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:31.538272 5806 hybrid_clock.cc:648] HybridClock initialized: now 1786303831538264 us; error 37 us; skew 500 ppm
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:31.540657 5806 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:31.541086 5806 webserver.cc:460] Webserver started at http://127.31.57.195:34705/ using document root <none> and password file <none>
I20260809 19:30:31.541311 5806 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:31.541376 5806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:31.541520 5806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:31.542465 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "a8db424967f74243a00e649d1f097270"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "e9c49fb16a60b0d63ce58706db93ab47"
server_key_iv: "2127d85ce1bce735e98909168c8cf2cb"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.542804 5806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "a8db424967f74243a00e649d1f097270"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "e9c49fb16a60b0d63ce58706db93ab47"
server_key_iv: "2127d85ce1bce735e98909168c8cf2cb"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.544178 5806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:31.544895 5840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.545059 5806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:31.545154 5806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "a8db424967f74243a00e649d1f097270"
format_stamp: "Formatted at 2026-08-09 19:30:31 on dist-test-slave-x4qh"
server_key: "e9c49fb16a60b0d63ce58706db93ab47"
server_key_iv: "2127d85ce1bce735e98909168c8cf2cb"
server_key_version: "encryptionkey@0"
I20260809 19:30:31.545233 5806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:31.556478 5806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:31.558635 5806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:31.558781 5806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:31.558976 5806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:31.559238 5806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:31.559301 5806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.559340 5806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:31.559365 5806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:31.565122 5806 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:41923
I20260809 19:30:31.565168 6013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:41923 every 8 connection(s)
I20260809 19:30:31.565440 5806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:31.565727 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 5806
I20260809 19:30:31.565866 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:31.566052 31975 external_mini_cluster.cc:1428] Setting key c3eeb59b404a9afc16cfad2cf1b9816d
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:31.574451 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.566088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54101 (local address 127.31.57.254:41717)
0809 19:30:31.566141 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.566143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.566366 (+ 223us) server_negotiation.cc:407] Connection header received
0809 19:30:31.567463 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.567464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.567475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.567539 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.568094 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.568582 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.569505 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.569681 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.569730 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.571761 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.571763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.571763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.571776 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.573050 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.573474 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.573475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.573475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.573535 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.573817 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.573818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.573818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.573934 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.574012 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.574280 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.574367 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:30:31.574630 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.566066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41717 (local address 127.31.57.195:54101)
0809 19:30:31.566221 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:31.566270 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:31.567322 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:31.567618 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:31.567621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:31.567854 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:31.568021 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.568025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.568657 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.568658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.569430 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:31.569432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.569751 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.569751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:31.569852 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.570222 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:31.570230 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:31.571659 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:31.573172 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.573179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.573180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.573395 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.573604 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:31.573604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:31.573605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:31.573727 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:31.574005 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:31.574011 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:31.574151 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 19:30:31.574256 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:30:31.574398 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260809 19:30:31.575112 6014 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41717
I20260809 19:30:31.575186 6014 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:31.575410 6014 heartbeater.cc:511] Master 127.31.57.254:41717 requested a full tablet report, sending...
I20260809 19:30:31.575807 5306 ts_manager.cc:194] Registered new tserver with Master: a8db424967f74243a00e649d1f097270 (127.31.57.195:41923)
I20260809 19:30:31.576154 5306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:54101
I20260809 19:30:31.577626 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:31.600374 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.593346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.31.57.193:37727)
0809 19:30:31.593525 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.593529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.593539 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:31.593563 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.593565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.593578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.593644 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.594207 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.594893 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.595595 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.595874 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.595926 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.597969 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.597976 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.597977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.597983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.599487 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.599756 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.599759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.599759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.599816 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.599954 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.599954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.599954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.600056 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.600147 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.600202 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.600258 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[5231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:31.607796 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:31.601561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.31.57.254:41717)
0809 19:30:31.601646 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:31.601648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:31.601668 (+ 20us) server_negotiation.cc:407] Connection header received
0809 19:30:31.601732 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:31.601733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:31.601742 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:31.601784 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:31.602201 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.602670 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.603302 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:31.603535 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:31.603587 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:31.605652 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:31.605653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:31.605653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:31.605658 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:31.606818 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.607110 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.607111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.607111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.607159 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:31.607328 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:31.607329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:31.607329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:31.607503 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:31.607576 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:31.607651 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 19:30:31.607708 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 19:30:31.608153 5306 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32786
I20260809 19:30:31.608471 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 5400
I20260809 19:30:31.630749 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 5596
I20260809 19:30:31.639328 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 5806
I20260809 19:30:31.649745 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 5246
2026-08-09T19:30:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1939 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[6033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[6033](info): set up 2 sockets
Aug 09 19:30:31 dist-test-slave-x4qh krb5kdc[6033](info): commencing operation
krb5kdc: starting...
W20260809 19:30:32.816365 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:32 dist-test-slave-x4qh krb5kdc[6033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:32.925429 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46219
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:36931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:33.001487 6050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:33.001698 6050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:33.001730 6050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:33.002768 6050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:33.002826 6050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:33.002848 6050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:33.002867 6050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:33.002897 6050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:33.004369 6050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:33.004621 6050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:33.004825 6050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:33.009284 6057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.009325 6059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.009302 6064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.009618 6050 server_base.cc:1034] running on GCE node
I20260809 19:30:33.009807 6050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:33.010042 6050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:33.011193 6050 hybrid_clock.cc:648] HybridClock initialized: now 1786303833011181 us; error 38 us; skew 500 ppm
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:33.016705 6050 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:33.017031 6050 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:33.017077 6050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:33.017160 6050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:33.018035 6050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "5ba19169500b4ec8a1c779cfa97edbaa"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "0158f49b9a9870793acdb6643d06a0f5"
server_key_iv: "c3986f72fd296efe845973cc0c96c317"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.018314 6050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "5ba19169500b4ec8a1c779cfa97edbaa"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "0158f49b9a9870793acdb6643d06a0f5"
server_key_iv: "c3986f72fd296efe845973cc0c96c317"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.019477 6050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:33.020133 6073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.020321 6050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:33.020404 6050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "5ba19169500b4ec8a1c779cfa97edbaa"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "0158f49b9a9870793acdb6643d06a0f5"
server_key_iv: "c3986f72fd296efe845973cc0c96c317"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.020453 6050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:33.028241 6050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:33.029716 6050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:33.029820 6050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:33.033530 6050 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:46219
I20260809 19:30:33.033556 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:46219 every 8 connection(s)
I20260809 19:30:33.033826 6050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:33.034765 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:33.036934 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: Bootstrap starting.
I20260809 19:30:33.037480 6144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:33.037755 6144 log.cc:824] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:33.038363 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: No bootstrap required, opened a new log
I20260809 19:30:33.039806 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 6050
I20260809 19:30:33.039901 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:33.040058 31975 external_mini_cluster.cc:1428] Setting key 2b72deb1b0b25a5310e79c4e172c8adf
I20260809 19:30:33.039983 6144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } }
I20260809 19:30:33.040127 6144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:33.040170 6144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba19169500b4ec8a1c779cfa97edbaa, State: Initialized, Role: FOLLOWER
I20260809 19:30:33.040313 6144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } }
I20260809 19:30:33.040398 6144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:33.040431 6144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:33.040477 6144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:33.041468 6144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } }
I20260809 19:30:33.041555 6144 leader_election.cc:302] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ba19169500b4ec8a1c779cfa97edbaa; no voters:
I20260809 19:30:33.041738 6144 leader_election.cc:288] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:33.041767 6151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:33.041878 6151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [term 1 LEADER]: Becoming Leader. State: Replica: 5ba19169500b4ec8a1c779cfa97edbaa, State: Running, Role: LEADER
I20260809 19:30:33.041957 6151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } }
I20260809 19:30:33.041996 6144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:33.042265 6154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } } }
I20260809 19:30:33.042490 6154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:33.042431 6152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ba19169500b4ec8a1c779cfa97edbaa. Latest consensus state: current_term: 1 leader_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba19169500b4ec8a1c779cfa97edbaa" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46219 } } }
I20260809 19:30:33.042729 6152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:33.042889 6163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:33.043450 6163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:33.046176 6163 catalog_manager.cc:1347] Generated new cluster ID: 46002b0095624cf5a181e0c754274c68
I20260809 19:30:33.046231 6163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:33.050761 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.040458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.31.57.254:46219)
0809 19:30:33.040660 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.040685 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.040717 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:30:33.040852 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.040859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.040932 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.041086 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:33.041713 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.043081 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.044188 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.044777 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.044900 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.047229 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:33.047233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:33.047233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:33.047244 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:33.049302 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.049652 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.049655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.049656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.049767 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.049966 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.049967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.049967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.050247 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:33.050395 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.050475 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.050598 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260809 19:30:33.052865 6163 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:33.053352 6163 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:33.055622 6163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: Generated new TSK 0
I20260809 19:30:33.055776 6163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:33.081727 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:36931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:33.156610 6190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:33.156806 6190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:33.156828 6190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:33.157866 6190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:33.157903 6190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:33.157959 6190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:33.159251 6190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:33.159500 6190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:33.159710 6190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:30:33.164490 6190 server_base.cc:1034] running on GCE node
W20260809 19:30:33.164425 6197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.164425 6200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.164440 6198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.164891 6190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:33.165159 6190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:33.166304 6190 hybrid_clock.cc:648] HybridClock initialized: now 1786303833166285 us; error 37 us; skew 500 ppm
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:33.168743 6190 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:33.169060 6190 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:33.169132 6190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:33.169286 6190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:33.170215 6190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "06e66c4418a4499c8ce35478863dfa34"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "f96b1fd5f6e6e6a9ae7d436a06e450f8"
server_key_iv: "60b984549f63537a3a33dd95838293c7"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.170588 6190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "06e66c4418a4499c8ce35478863dfa34"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "f96b1fd5f6e6e6a9ae7d436a06e450f8"
server_key_iv: "60b984549f63537a3a33dd95838293c7"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.171941 6190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:33.172628 6210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.172791 6190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:33.172919 6190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "06e66c4418a4499c8ce35478863dfa34"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "f96b1fd5f6e6e6a9ae7d436a06e450f8"
server_key_iv: "60b984549f63537a3a33dd95838293c7"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.172997 6190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:33.194932 6190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:33.196597 6190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:33.196748 6190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:33.196956 6190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:33.197232 6190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:33.197291 6190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.197324 6190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:33.197352 6190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.203044 6190 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:36699
I20260809 19:30:33.203131 6385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:36699 every 8 connection(s)
I20260809 19:30:33.203394 6190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:33.206075 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 6190
I20260809 19:30:33.206182 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:33.206346 31975 external_mini_cluster.cc:1428] Setting key d34135ffdccccc83845769402cce7ad2
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:33.212989 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.204069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46219 (local address 127.31.57.193:44667)
0809 19:30:33.204252 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.204289 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.204706 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.204912 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.204916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.205138 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:33.205581 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.205585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.206205 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.206206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.207157 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.207160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.207593 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.207595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.207753 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.208164 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:33.208173 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:33.209903 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:33.211710 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.211716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.211717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.211921 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.212140 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.212141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.212141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.212230 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.212519 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:33.212523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:33.212622 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.212698 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.212827 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260809 19:30:33.213032 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.204105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:44667 (local address 127.31.57.254:46219)
0809 19:30:33.204167 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.204169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.204380 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:30:33.204787 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.204788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.204797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.204850 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:33.205649 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.206094 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.207253 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.207515 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.207570 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.210022 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:33.210023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:33.210024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:33.210033 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:33.211628 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.212008 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.212009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.212009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.212071 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.212305 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.212306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.212306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.212448 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:33.212527 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.212761 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.212893 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:30:33.213644 6386 heartbeater.cc:347] Connected to a master server at 127.31.57.254:46219
I20260809 19:30:33.213749 6386 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:33.213891 6386 heartbeater.cc:511] Master 127.31.57.254:46219 requested a full tablet report, sending...
I20260809 19:30:33.214491 6095 ts_manager.cc:194] Registered new tserver with Master: 06e66c4418a4499c8ce35478863dfa34 (127.31.57.193:36699)
I20260809 19:30:33.215404 6095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:44667
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:33.232288 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:36931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:33.307792 6393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:33.308027 6393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:33.308061 6393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:33.309095 6393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:33.309156 6393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:33.309198 6393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:33.310633 6393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:33.310887 6393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:33.311121 6393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:33.315902 6398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.315891 6399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.316216 6402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.316969 6393 server_base.cc:1034] running on GCE node
I20260809 19:30:33.317150 6393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:33.317396 6393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:33.318552 6393 hybrid_clock.cc:648] HybridClock initialized: now 1786303833318530 us; error 33 us; skew 500 ppm
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:33.320849 6393 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:33.321211 6393 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:33.321278 6393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:33.321415 6393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:33.322348 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "d2e2321678874277ad383fad645b001c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "e1cb75e6e0c4f550229f1500a8806abb"
server_key_iv: "97a3221d200626a3c323adbbb03d0bc1"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.322681 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "d2e2321678874277ad383fad645b001c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "e1cb75e6e0c4f550229f1500a8806abb"
server_key_iv: "97a3221d200626a3c323adbbb03d0bc1"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.324214 6393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:33.324961 6412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.325163 6393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:33.325256 6393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "d2e2321678874277ad383fad645b001c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "e1cb75e6e0c4f550229f1500a8806abb"
server_key_iv: "97a3221d200626a3c323adbbb03d0bc1"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.325336 6393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:33.348971 6393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:33.350636 6393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:33.350790 6393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:33.350992 6393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:33.351270 6393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:33.351330 6393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.351362 6393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:33.351416 6393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.356941 6393 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:40063
I20260809 19:30:33.356993 6588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:40063 every 8 connection(s)
I20260809 19:30:33.357239 6393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:33.365056 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.357857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:60505 (local address 127.31.57.254:46219)
0809 19:30:33.357930 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.357932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.358100 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:30:33.358511 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.358512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.358522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.358566 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:33.359328 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.359775 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.360617 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.360775 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.360820 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.362667 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:33.362668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:33.362669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:33.362674 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:33.363918 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.364307 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.364308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.364308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.364364 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.364567 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.364568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.364568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.364657 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:33.364703 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.364932 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.364996 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:30:33.365255 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.357819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46219 (local address 127.31.57.194:60505)
0809 19:30:33.357986 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.358017 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.358434 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.358619 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.358622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.358844 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:33.359258 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.359263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.359850 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.359851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.360546 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.360548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.360844 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.360844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.360934 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.361405 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:33.361413 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:33.362584 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:33.363991 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.363997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.363999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.364221 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.364423 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.364423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.364424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.364513 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.364711 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:33.364716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:33.364812 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.364890 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.365020 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260809 19:30:33.365849 6591 heartbeater.cc:347] Connected to a master server at 127.31.57.254:46219
I20260809 19:30:33.365950 6591 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:33.366098 6591 heartbeater.cc:511] Master 127.31.57.254:46219 requested a full tablet report, sending...
I20260809 19:30:33.366437 6095 ts_manager.cc:194] Registered new tserver with Master: d2e2321678874277ad383fad645b001c (127.31.57.194:40063)
I20260809 19:30:33.366739 6095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:60505
I20260809 19:30:33.366751 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 6393
I20260809 19:30:33.366871 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:33.367162 31975 external_mini_cluster.cc:1428] Setting key cbe15fcccaeedf7a08b53f2a82aa4091
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:33.389567 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:36931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:33.462041 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:33.462199 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:33.462218 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:33.463289 6597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:33.463327 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:33.463353 6597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:33.464762 6597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:33.464916 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:33.465063 6597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:33.469492 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.469594 6597 server_base.cc:1034] running on GCE node
W20260809 19:30:33.469503 6609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.469502 6604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.469939 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:33.470187 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:33.471341 6597 hybrid_clock.cc:648] HybridClock initialized: now 1786303833471333 us; error 37 us; skew 500 ppm
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:33.473585 6597 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:33.473958 6597 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:33.474026 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:33.474156 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:33.475221 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "ccae58f6d88940f2a44104cff8a99f5c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "4bbd1a1ea3f8b79e0fa4558f2b790554"
server_key_iv: "6640f2ce579f4458635d42681b0d7ce2"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.475656 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "ccae58f6d88940f2a44104cff8a99f5c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "4bbd1a1ea3f8b79e0fa4558f2b790554"
server_key_iv: "6640f2ce579f4458635d42681b0d7ce2"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.476969 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:33.477617 6620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.477787 6597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.477880 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "ccae58f6d88940f2a44104cff8a99f5c"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "4bbd1a1ea3f8b79e0fa4558f2b790554"
server_key_iv: "6640f2ce579f4458635d42681b0d7ce2"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.477968 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:33.494239 6597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:33.495853 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:33.495997 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:33.496191 6597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:33.496466 6597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:33.496522 6597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.496558 6597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:33.496603 6597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:33.502344 6597 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:44649
I20260809 19:30:33.502377 6793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:44649 every 8 connection(s)
I20260809 19:30:33.502614 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:33.503564 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 6597
I20260809 19:30:33.503674 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:33.503837 31975 external_mini_cluster.cc:1428] Setting key 6197303489d29db4258e7fa501532f7e
Aug 09 19:30:33 dist-test-slave-x4qh krb5kdc[6033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303833, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:33.513383 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.503282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:33189 (local address 127.31.57.254:46219)
0809 19:30:33.503361 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.503364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.503588 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:30:33.504147 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.504148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.504160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.504210 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:33.505182 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.505744 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.507234 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.507564 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.507625 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.510328 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:33.510330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:33.510330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:33.510337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:33.511894 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.512335 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.512336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.512337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.512398 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.512666 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.512667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.512667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.512805 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:33.512852 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.513203 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.513290 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:30:33.513863 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.503254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46219 (local address 127.31.57.195:33189)
0809 19:30:33.503470 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.503500 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.504062 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.504270 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.504273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.504595 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:33.505086 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.505091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.505858 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.505860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.507129 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.507133 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.507645 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.507647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.507792 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.508322 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:33.508331 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:33.510225 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:33.511984 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.511990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.511992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.512245 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.512470 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:33.512471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:33.512471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:33.512583 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:33.512899 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:33.512905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:33.513037 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.513468 (+ 431us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.513625 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
I20260809 19:30:33.516901 6796 heartbeater.cc:347] Connected to a master server at 127.31.57.254:46219
I20260809 19:30:33.517006 6796 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:33.517148 6796 heartbeater.cc:511] Master 127.31.57.254:46219 requested a full tablet report, sending...
I20260809 19:30:33.517521 6095 ts_manager.cc:194] Registered new tserver with Master: ccae58f6d88940f2a44104cff8a99f5c (127.31.57.195:44649)
I20260809 19:30:33.517863 6095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:33189
I20260809 19:30:33.527755 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:30:33.534301 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.529055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.31.57.254:46219)
0809 19:30:33.529125 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.529127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.529208 (+ 81us) server_negotiation.cc:407] Connection header received
0809 19:30:33.529250 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.529251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.529258 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.529311 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:33.529810 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.530234 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.530934 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.531089 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.531133 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.532127 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:33.532128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:33.532128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:33.532133 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:33.533410 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.533690 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.533691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.533691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.533738 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:33.533886 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:33.533886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:33.533886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:33.533998 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:33.534063 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.534148 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.534201 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:30:33.535601 6095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:33.537031 6095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:33.542527 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.538960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:40063 (local address 127.0.0.1:60728)
0809 19:30:33.539099 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.539133 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.539192 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.539387 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.539389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.539391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:33.539447 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.539449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540611 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.540612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.541424 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.541426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542246 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.542246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.542317 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.542329 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.542400 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.542444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260809 19:30:33.542547 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.539051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.31.57.194:40063)
0809 19:30:33.539213 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.539217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.539228 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:30:33.539265 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.539267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.539280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.539358 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:33.539502 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540508 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.541512 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542178 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.542244 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.542291 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.542419 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.542456 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260809 19:30:33.543046 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.538983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36699 (local address 127.0.0.1:40104)
0809 19:30:33.539459 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.539468 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.539524 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.539765 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.539788 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.539790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:33.539843 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.539844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540731 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.540731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.541787 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.541789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542797 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.542798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.542864 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.542868 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.542920 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.542968 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260809 19:30:33.543208 6808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.538983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:44649 (local address 127.0.0.1:32998)
0809 19:30:33.539595 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.539603 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.539649 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.539806 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.539807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.539808 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:33.539842 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.539844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540924 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.540925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.541875 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.541877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542860 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.542878 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.542947 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.542952 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.542986 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.543033 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":4,"thread_start_us":52,"threads_started":1}
I20260809 19:30:33.543329 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.539147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.31.57.195:44649)
0809 19:30:33.539446 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.539450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.539666 (+ 216us) server_negotiation.cc:407] Connection header received
0809 19:30:33.539704 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.539707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.539720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.539858 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:33.539917 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540818 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.541940 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542690 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.543090 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.543184 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.543211 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.543246 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":78,"threads_started":1}
I20260809 19:30:33.543349 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.539182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.31.57.193:36699)
0809 19:30:33.539350 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.539353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.539519 (+ 166us) server_negotiation.cc:407] Connection header received
0809 19:30:33.539590 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.539592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.539609 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.539681 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:33.539927 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.540644 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.541896 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.542643 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.542707 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.542758 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.543212 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.543264 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
I20260809 19:30:33.543401 6488 tablet_service.cc:1467] Processing CreateTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 (DEFAULT_TABLE table=test-table [id=9ba5be19b5674aeebdbe13653268ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:33.543732 6488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4274c3b07e949d6a3a15176cd2ed205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:33.546177 6815 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Bootstrap starting.
I20260809 19:30:33.546734 6815 tablet_bootstrap.cc:654] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:33.546880 6691 tablet_service.cc:1467] Processing CreateTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 (DEFAULT_TABLE table=test-table [id=9ba5be19b5674aeebdbe13653268ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:33.547183 6691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4274c3b07e949d6a3a15176cd2ed205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:33.547163 6279 tablet_service.cc:1467] Processing CreateTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 (DEFAULT_TABLE table=test-table [id=9ba5be19b5674aeebdbe13653268ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:33.547333 6815 log.cc:824] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:33.547557 6279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4274c3b07e949d6a3a15176cd2ed205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:33.548164 6815 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: No bootstrap required, opened a new log
I20260809 19:30:33.548272 6815 ts_tablet_manager.cc:1397] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:33.549811 6818 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Bootstrap starting.
I20260809 19:30:33.550479 6818 tablet_bootstrap.cc:654] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:33.550386 6815 raft_consensus.cc:348] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.550552 6815 raft_consensus.cc:374] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:33.550594 6815 raft_consensus.cc:729] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e2321678874277ad383fad645b001c, State: Initialized, Role: FOLLOWER
I20260809 19:30:33.550702 6815 consensus_queue.cc:260] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.550956 6815 ts_tablet_manager.cc:1428] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:30:33.550961 6819 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Bootstrap starting.
I20260809 19:30:33.551077 6591 heartbeater.cc:503] Master 127.31.57.254:46219 was elected leader, sending a full tablet report...
I20260809 19:30:33.551252 6818 log.cc:824] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:33.551666 6819 tablet_bootstrap.cc:654] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:33.552151 6818 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: No bootstrap required, opened a new log
I20260809 19:30:33.552223 6818 ts_tablet_manager.cc:1397] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:33.552249 6819 log.cc:824] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:33.552930 6819 tablet_bootstrap.cc:492] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: No bootstrap required, opened a new log
I20260809 19:30:33.552999 6819 ts_tablet_manager.cc:1397] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:30:33.554150 6818 raft_consensus.cc:348] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.554318 6818 raft_consensus.cc:374] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:33.554353 6818 raft_consensus.cc:729] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccae58f6d88940f2a44104cff8a99f5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:33.554450 6818 consensus_queue.cc:260] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.554728 6818 ts_tablet_manager.cc:1428] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:30:33.554754 6796 heartbeater.cc:503] Master 127.31.57.254:46219 was elected leader, sending a full tablet report...
I20260809 19:30:33.555020 6819 raft_consensus.cc:348] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.555136 6819 raft_consensus.cc:374] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:33.555195 6819 raft_consensus.cc:729] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e66c4418a4499c8ce35478863dfa34, State: Initialized, Role: FOLLOWER
I20260809 19:30:33.555312 6819 consensus_queue.cc:260] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.555595 6819 ts_tablet_manager.cc:1428] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:30:33.555668 6386 heartbeater.cc:503] Master 127.31.57.254:46219 was elected leader, sending a full tablet report...
W20260809 19:30:33.607795 6593 tablet.cc:2369] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:33.704051 6387 tablet.cc:2369] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:33.753177 6798 tablet.cc:2369] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:33.776335 6826 raft_consensus.cc:482] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:33.776444 6826 raft_consensus.cc:504] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.776791 6826 leader_election.cc:288] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e2321678874277ad383fad645b001c (127.31.57.194:40063), ccae58f6d88940f2a44104cff8a99f5c (127.31.57.195:44649)
I20260809 19:30:33.779552 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.777035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:58359 (local address 127.31.57.195:44649)
0809 19:30:33.777093 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.777096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.777180 (+ 84us) server_negotiation.cc:407] Connection header received
0809 19:30:33.777253 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.777254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.777261 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.777337 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:33.777536 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.777944 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.778848 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.779179 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.779263 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.779276 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.779445 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.779486 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:30:33.779569 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.776952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:44649 (local address 127.31.57.193:58359)
0809 19:30:33.777108 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.777118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.777198 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.777361 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.777361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.777363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:33.777461 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.777463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.778060 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.778061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.778719 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.778721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.779264 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.779265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.779332 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.779336 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.779437 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.779480 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260809 19:30:33.779567 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.777047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:42879 (local address 127.31.57.194:40063)
0809 19:30:33.777105 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.777106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.777146 (+ 40us) server_negotiation.cc:407] Connection header received
0809 19:30:33.777224 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.777225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.777235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.777302 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:33.777467 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.777920 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.778820 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.779181 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.779286 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.779294 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.779478 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.779513 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:30:33.779590 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.776952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:40063 (local address 127.31.57.193:42879)
0809 19:30:33.777086 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.777097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:33.777154 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:33.777309 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:33.777310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:33.777312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:33.777395 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.777397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.778003 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.778004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.778712 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:33.778713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.779300 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.779301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:33.779417 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.779421 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:33.779473 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:30:33.779507 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260809 19:30:33.779897 6519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4274c3b07e949d6a3a15176cd2ed205" candidate_uuid: "06e66c4418a4499c8ce35478863dfa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e2321678874277ad383fad645b001c" is_pre_election: true
I20260809 19:30:33.779897 6718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4274c3b07e949d6a3a15176cd2ed205" candidate_uuid: "06e66c4418a4499c8ce35478863dfa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccae58f6d88940f2a44104cff8a99f5c" is_pre_election: true
I20260809 19:30:33.780061 6718 raft_consensus.cc:2454] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06e66c4418a4499c8ce35478863dfa34 in term 0.
I20260809 19:30:33.780061 6519 raft_consensus.cc:2454] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06e66c4418a4499c8ce35478863dfa34 in term 0.
I20260809 19:30:33.780293 6215 leader_election.cc:302] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e66c4418a4499c8ce35478863dfa34, d2e2321678874277ad383fad645b001c; no voters:
I20260809 19:30:33.780445 6826 raft_consensus.cc:2781] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:33.780499 6826 raft_consensus.cc:482] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:33.780519 6826 raft_consensus.cc:3037] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:33.781318 6826 raft_consensus.cc:504] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.781478 6826 leader_election.cc:288] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [CANDIDATE]: Term 1 election: Requested vote from peers d2e2321678874277ad383fad645b001c (127.31.57.194:40063), ccae58f6d88940f2a44104cff8a99f5c (127.31.57.195:44649)
I20260809 19:30:33.781653 6519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4274c3b07e949d6a3a15176cd2ed205" candidate_uuid: "06e66c4418a4499c8ce35478863dfa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e2321678874277ad383fad645b001c"
I20260809 19:30:33.781728 6519 raft_consensus.cc:3037] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:33.781670 6718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4274c3b07e949d6a3a15176cd2ed205" candidate_uuid: "06e66c4418a4499c8ce35478863dfa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccae58f6d88940f2a44104cff8a99f5c"
I20260809 19:30:33.781790 6718 raft_consensus.cc:3037] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:33.782459 6519 raft_consensus.cc:2454] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e66c4418a4499c8ce35478863dfa34 in term 1.
I20260809 19:30:33.782629 6718 raft_consensus.cc:2454] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e66c4418a4499c8ce35478863dfa34 in term 1.
I20260809 19:30:33.782640 6215 leader_election.cc:302] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e66c4418a4499c8ce35478863dfa34, d2e2321678874277ad383fad645b001c; no voters:
I20260809 19:30:33.782760 6826 raft_consensus.cc:2781] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:33.782989 6826 raft_consensus.cc:686] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 1 LEADER]: Becoming Leader. State: Replica: 06e66c4418a4499c8ce35478863dfa34, State: Running, Role: LEADER
I20260809 19:30:33.783089 6826 consensus_queue.cc:237] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } }
I20260809 19:30:33.783900 6095 catalog_manager.cc:5495] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06e66c4418a4499c8ce35478863dfa34 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "06e66c4418a4499c8ce35478863dfa34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e66c4418a4499c8ce35478863dfa34" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:33.826260 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.824296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.31.57.193:36699)
0809 19:30:33.824376 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:33.824378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:33.824388 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:33.824422 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:33.824423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:33.824439 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:33.824496 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:33.824605 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.825038 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.825689 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:33.825851 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:33.825896 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:33.826016 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:33.826109 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:33.826164 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:33.826195 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:30:33.826210 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:30:33.828029 6718 raft_consensus.cc:1261] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 1 FOLLOWER]: Refusing update from remote peer 06e66c4418a4499c8ce35478863dfa34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:33.828063 6519 raft_consensus.cc:1261] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 1 FOLLOWER]: Refusing update from remote peer 06e66c4418a4499c8ce35478863dfa34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:33.828461 6832 consensus_queue.cc:1035] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccae58f6d88940f2a44104cff8a99f5c" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 44649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:33.828610 6826 consensus_queue.cc:1035] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e2321678874277ad383fad645b001c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:33.832775 6095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:33.832878 6095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:33.833673 6095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ba19169500b4ec8a1c779cfa97edbaa: Sending DeleteTablet for 3 replicas of tablet f4274c3b07e949d6a3a15176cd2ed205
I20260809 19:30:33.834007 6691 tablet_service.cc:1514] Processing DeleteTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:33 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:32998
I20260809 19:30:33.834031 6279 tablet_service.cc:1514] Processing DeleteTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:33 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:40104
I20260809 19:30:33.834046 6488 tablet_service.cc:1514] Processing DeleteTablet for tablet f4274c3b07e949d6a3a15176cd2ed205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:33 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:60728
I20260809 19:30:33.834242 6856 tablet_replica.cc:330] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34: stopping tablet replica
I20260809 19:30:33.834318 6856 raft_consensus.cc:2229] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:33.834451 6854 tablet_replica.cc:330] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: stopping tablet replica
I20260809 19:30:33.834512 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 6190
I20260809 19:30:33.834414 6856 raft_consensus.cc:2258] T f4274c3b07e949d6a3a15176cd2ed205 P 06e66c4418a4499c8ce35478863dfa34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:33.837139 6854 raft_consensus.cc:2229] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:33.837301 6854 raft_consensus.cc:2258] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:33.837517 6853 tablet_replica.cc:330] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: stopping tablet replica
I20260809 19:30:33.837627 6853 raft_consensus.cc:2229] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:33.837749 6853 raft_consensus.cc:2258] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:33.838181 6854 ts_tablet_manager.cc:1905] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:33.840375 6854 ts_tablet_manager.cc:1918] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:33.840476 6854 log.cc:1196] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/f4274c3b07e949d6a3a15176cd2ed205
I20260809 19:30:33.840502 6853 ts_tablet_manager.cc:1905] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:33.840848 6854 ts_tablet_manager.cc:1939] T f4274c3b07e949d6a3a15176cd2ed205 P d2e2321678874277ad383fad645b001c: Deleting consensus metadata
I20260809 19:30:33.841375 6078 catalog_manager.cc:4841] TS d2e2321678874277ad383fad645b001c (127.31.57.194:40063): tablet f4274c3b07e949d6a3a15176cd2ed205 (table test-table [id=9ba5be19b5674aeebdbe13653268ac08]) successfully deleted
I20260809 19:30:33.842325 6853 ts_tablet_manager.cc:1918] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:33.842394 6853 log.cc:1196] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/f4274c3b07e949d6a3a15176cd2ed205
I20260809 19:30:33.842674 6853 ts_tablet_manager.cc:1939] T f4274c3b07e949d6a3a15176cd2ed205 P ccae58f6d88940f2a44104cff8a99f5c: Deleting consensus metadata
I20260809 19:30:33.843070 6075 catalog_manager.cc:4841] TS ccae58f6d88940f2a44104cff8a99f5c (127.31.57.195:44649): tablet f4274c3b07e949d6a3a15176cd2ed205 (table test-table [id=9ba5be19b5674aeebdbe13653268ac08]) successfully deleted
W20260809 19:30:33.856314 6074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:36699 (error 108)
I20260809 19:30:33.856609 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 6393
I20260809 19:30:33.856778 6808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:33.856638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36699 (local address 127.0.0.1:40134)
0809 19:30:33.856698 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:33.856743 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 19:30:33.858002 6074 catalog_manager.cc:4584] TS 06e66c4418a4499c8ce35478863dfa34 (127.31.57.193:36699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4274c3b07e949d6a3a15176cd2ed205: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36699: connect: Connection refused (error 111)
I20260809 19:30:33.865942 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 6597
I20260809 19:30:33.881098 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 6050
2026-08-09T19:30:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2228 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T19:30:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:33Z Disabled control of system clock
I20260809 19:30:33.910815 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:44627
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:44627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:30:33.987269 6863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:33.987536 6863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:33.987561 6863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:33.988652 6863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:33.988684 6863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:33.988698 6863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:33.988709 6863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:33.988721 6863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:33.989926 6863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:44627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:44627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:33.990123 6863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:33.990337 6863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:33.994880 6869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:33.994889 6871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.995112 6863 server_base.cc:1034] running on GCE node
W20260809 19:30:33.994899 6875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:33.995512 6863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:33.995801 6863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:33.996963 6863 hybrid_clock.cc:648] HybridClock initialized: now 1786303833996960 us; error 41 us; skew 500 ppm
I20260809 19:30:33.998109 6863 webserver.cc:460] Webserver started at http://127.31.57.254:46731/ using document root <none> and password file <none>
I20260809 19:30:33.998348 6863 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:33.998413 6863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:33.998539 6863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:33.999544 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "36a8413dd5d64dc0bf1374970a0adc06"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "794e56fa59375ea72e8583585b8ec280"
server_key_iv: "28b606f0b24a59e3f0bdcbeb2b883322"
server_key_version: "encryptionkey@0"
I20260809 19:30:33.999881 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "36a8413dd5d64dc0bf1374970a0adc06"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "794e56fa59375ea72e8583585b8ec280"
server_key_iv: "28b606f0b24a59e3f0bdcbeb2b883322"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.001104 6863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:34.001809 6885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.001976 6863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:34.002072 6863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "36a8413dd5d64dc0bf1374970a0adc06"
format_stamp: "Formatted at 2026-08-09 19:30:33 on dist-test-slave-x4qh"
server_key: "794e56fa59375ea72e8583585b8ec280"
server_key_iv: "28b606f0b24a59e3f0bdcbeb2b883322"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.002151 6863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.005576 6863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.005829 6863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.005972 6863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.009624 6863 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:44627
I20260809 19:30:34.009651 6976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:44627 every 8 connection(s)
I20260809 19:30:34.009954 6863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:34.010584 6979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.013190 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: Bootstrap starting.
I20260809 19:30:34.013733 6979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.014024 6979 log.cc:824] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.014662 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 6863
I20260809 19:30:34.014725 6979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: No bootstrap required, opened a new log
I20260809 19:30:34.014771 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:34.014956 31975 external_mini_cluster.cc:1428] Setting key 53647cd0731d748d04afa97271a4e8aa
I20260809 19:30:34.016494 6979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } }
I20260809 19:30:34.016640 6979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.016685 6979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a8413dd5d64dc0bf1374970a0adc06, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.016805 6979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } }
I20260809 19:30:34.016924 6979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:34.016963 6979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:34.016999 6979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.017534 6979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } }
I20260809 19:30:34.017653 6979 leader_election.cc:302] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a8413dd5d64dc0bf1374970a0adc06; no voters:
I20260809 19:30:34.017803 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.015448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.31.57.254:44627)
0809 19:30:34.015664 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.015672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.015705 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:30:34.016482 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.016488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.016536 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.016671 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.017358 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.017362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.017362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.017366 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.017411 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.017422 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.017497 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.017564 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.017616 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260809 19:30:34.017853 6979 leader_election.cc:288] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:34.018096 6986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:34.018102 6979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:34.018214 6986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [term 1 LEADER]: Becoming Leader. State: Replica: 36a8413dd5d64dc0bf1374970a0adc06, State: Running, Role: LEADER
I20260809 19:30:34.018286 6986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } }
I20260809 19:30:34.018633 6990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a8413dd5d64dc0bf1374970a0adc06. Latest consensus state: current_term: 1 leader_uuid: "36a8413dd5d64dc0bf1374970a0adc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } } }
I20260809 19:30:34.018656 6987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a8413dd5d64dc0bf1374970a0adc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a8413dd5d64dc0bf1374970a0adc06" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44627 } } }
I20260809 19:30:34.018828 6990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.018846 6987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.019349 7018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:34.019886 7018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:34.021332 7018 catalog_manager.cc:1347] Generated new cluster ID: c9bdefb88d48422781002c9c7b7073fe
I20260809 19:30:34.021384 7018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:34.025315 7018 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:34.025472 7018 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:34.033740 7018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: Generated new TSK 0
I20260809 19:30:34.033915 7018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:30:34.037276 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--builtin_ntp_servers=127.31.57.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:30:34.119005 7027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:34.119201 7027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:34.119221 7027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:34.120311 7027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:34.120353 7027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:34.120375 7027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:34.121608 7027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:34.121831 7027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:34.122126 7027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:34.126786 7034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.126809 7036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.126929 7027 server_base.cc:1034] running on GCE node
W20260809 19:30:34.127091 7032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.127338 7027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:34.127624 7027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:34.128806 7027 hybrid_clock.cc:648] HybridClock initialized: now 1786303834128790 us; error 32 us; skew 500 ppm
I20260809 19:30:34.130167 7027 webserver.cc:460] Webserver started at http://127.31.57.193:35091/ using document root <none> and password file <none>
I20260809 19:30:34.130417 7027 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:34.130467 7027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:34.130573 7027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:34.131913 7027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "477d27c7487744cba0fa65387dbceb7a"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "38ae4da6575a4ca8df51059fa1d462d5"
server_key_iv: "659f184794f9ee874bb39cd4b4b4c6b4"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.132278 7027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "477d27c7487744cba0fa65387dbceb7a"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "38ae4da6575a4ca8df51059fa1d462d5"
server_key_iv: "659f184794f9ee874bb39cd4b4b4c6b4"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.133782 7027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:34.134567 7043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.134745 7027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:34.134829 7027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "477d27c7487744cba0fa65387dbceb7a"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "38ae4da6575a4ca8df51059fa1d462d5"
server_key_iv: "659f184794f9ee874bb39cd4b4b4c6b4"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.134889 7027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.138902 7027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.139137 7027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.139251 7027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.139454 7027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:34.139732 7027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:34.139787 7027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.139810 7027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:34.139829 7027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.145870 7027 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:45929
I20260809 19:30:34.145920 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:45929 every 8 connection(s)
I20260809 19:30:34.146205 7027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:34.149003 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.146873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:37549 (local address 127.31.57.254:44627)
0809 19:30:34.146933 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.146935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.147154 (+ 219us) server_negotiation.cc:407] Connection header received
0809 19:30:34.147663 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.147664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.147672 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.147718 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.148778 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.148780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.148780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.148785 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.148800 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.148804 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.148860 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.148927 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.148942 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:30:34.149175 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.146860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44627 (local address 127.31.57.193:37549)
0809 19:30:34.147053 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.147079 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.147559 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.147777 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.147780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.147789 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.148638 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.148639 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.148662 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.148687 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.148859 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.148864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.148867 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.148927 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.148982 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260809 19:30:34.149649 7236 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44627
I20260809 19:30:34.149716 7236 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:34.149874 7236 heartbeater.cc:511] Master 127.31.57.254:44627 requested a full tablet report, sending...
I20260809 19:30:34.150322 6912 ts_manager.cc:194] Registered new tserver with Master: 477d27c7487744cba0fa65387dbceb7a (127.31.57.193:45929)
I20260809 19:30:34.156232 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 7027
I20260809 19:30:34.156363 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:34.156491 31975 external_mini_cluster.cc:1428] Setting key 1284678c7d706682f57b2fb58bfe48ff
I20260809 19:30:34.157356 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--builtin_ntp_servers=127.31.57.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:30:34.235562 7243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:34.235769 7243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:34.235805 7243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:34.236933 7243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:34.236999 7243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:34.237044 7243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:34.238366 7243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:34.238612 7243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:34.238885 7243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:34.243829 7252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.243782 7248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.243782 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.244180 7243 server_base.cc:1034] running on GCE node
I20260809 19:30:34.244793 7243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:34.245081 7243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:34.246248 7243 hybrid_clock.cc:648] HybridClock initialized: now 1786303834246231 us; error 30 us; skew 500 ppm
I20260809 19:30:34.247516 7243 webserver.cc:460] Webserver started at http://127.31.57.194:36439/ using document root <none> and password file <none>
I20260809 19:30:34.247785 7243 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:34.247852 7243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:34.247987 7243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:34.249001 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "11cdd35040114484b978c8374cf5df13"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "4a9f9b2d9108d5a8b1bb767147058cbe"
server_key_iv: "7b8ba44ad4520b0fa7df86739c795254"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.249387 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "11cdd35040114484b978c8374cf5df13"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "4a9f9b2d9108d5a8b1bb767147058cbe"
server_key_iv: "7b8ba44ad4520b0fa7df86739c795254"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.250694 7243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:34.251348 7263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.251533 7243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:34.251634 7243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "11cdd35040114484b978c8374cf5df13"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "4a9f9b2d9108d5a8b1bb767147058cbe"
server_key_iv: "7b8ba44ad4520b0fa7df86739c795254"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.251725 7243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.255033 7243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.255277 7243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.255462 7243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.255703 7243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:34.256007 7243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:34.256062 7243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.256098 7243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:34.256127 7243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.262429 7243 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:34019
I20260809 19:30:34.262480 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:34019 every 8 connection(s)
I20260809 19:30:34.262784 7243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:34.265601 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.263509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:33263 (local address 127.31.57.254:44627)
0809 19:30:34.263566 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.263569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.263821 (+ 252us) server_negotiation.cc:407] Connection header received
0809 19:30:34.264220 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.264221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.264227 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.264285 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.265404 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.265405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.265406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.265408 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.265421 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.265425 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.265469 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.265529 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.265541 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:30:34.265767 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.263511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44627 (local address 127.31.57.194:33263)
0809 19:30:34.263707 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.263736 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.264139 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.264333 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.264336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.264353 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.265273 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.265274 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.265305 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.265331 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.265472 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.265474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.265476 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.265525 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.265568 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260809 19:30:34.266314 7455 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44627
I20260809 19:30:34.266410 7455 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:34.266570 7455 heartbeater.cc:511] Master 127.31.57.254:44627 requested a full tablet report, sending...
I20260809 19:30:34.266952 6912 ts_manager.cc:194] Registered new tserver with Master: 11cdd35040114484b978c8374cf5df13 (127.31.57.194:34019)
I20260809 19:30:34.272611 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 7243
I20260809 19:30:34.272689 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:34.272886 31975 external_mini_cluster.cc:1428] Setting key 60b5b107bb22ff829b915c5b6d2fa694
I20260809 19:30:34.273816 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--builtin_ntp_servers=127.31.57.212:44999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:30:34.350598 7459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:34.350800 7459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:34.350832 7459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:34.351864 7459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:34.351922 7459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:34.351961 7459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:34.353199 7459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:34.353448 7459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:34.353747 7459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:34.358325 7465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.358332 7467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.358577 7459 server_base.cc:1034] running on GCE node
W20260809 19:30:34.358636 7464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.358829 7459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:34.359025 7459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:34.360152 7459 hybrid_clock.cc:648] HybridClock initialized: now 1786303834360134 us; error 36 us; skew 500 ppm
I20260809 19:30:34.361188 7459 webserver.cc:460] Webserver started at http://127.31.57.195:38629/ using document root <none> and password file <none>
I20260809 19:30:34.361375 7459 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:34.361413 7459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:34.361495 7459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:34.362352 7459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "e2c55426726947fa80b402d1a4b2fa99"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "87ac35cdd8ffb077597c27ee06187b74"
server_key_iv: "4e2d3bbea2112778cf5a136644576ba8"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.362612 7459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "e2c55426726947fa80b402d1a4b2fa99"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "87ac35cdd8ffb077597c27ee06187b74"
server_key_iv: "4e2d3bbea2112778cf5a136644576ba8"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.363883 7459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:34.364537 7478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.364724 7459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:34.364802 7459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "e2c55426726947fa80b402d1a4b2fa99"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "87ac35cdd8ffb077597c27ee06187b74"
server_key_iv: "4e2d3bbea2112778cf5a136644576ba8"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.364853 7459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.367797 7459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.367997 7459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.368085 7459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.368224 7459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:34.368465 7459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:34.368494 7459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.368511 7459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:34.368523 7459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.373737 7459 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:42147
I20260809 19:30:34.373831 7665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:42147 every 8 connection(s)
I20260809 19:30:34.374123 7459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:34.376803 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.374854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:41137 (local address 127.31.57.254:44627)
0809 19:30:34.374917 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.374919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.375111 (+ 192us) server_negotiation.cc:407] Connection header received
0809 19:30:34.375547 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.375548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.375553 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.375603 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.376612 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.376613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.376613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.376617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.376631 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.376635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.376681 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.376736 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.376748 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:30:34.376979 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.374817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44627 (local address 127.31.57.195:41137)
0809 19:30:34.375000 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.375033 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.375457 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.375666 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.375668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.375676 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.376479 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.376479 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.376504 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.376528 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.376678 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.376680 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.376682 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.376729 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.376768 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260809 19:30:34.377408 7666 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44627
I20260809 19:30:34.377488 7666 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:34.377588 7666 heartbeater.cc:511] Master 127.31.57.254:44627 requested a full tablet report, sending...
I20260809 19:30:34.377876 6912 ts_manager.cc:194] Registered new tserver with Master: e2c55426726947fa80b402d1a4b2fa99 (127.31.57.195:42147)
I20260809 19:30:34.378844 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 7459
I20260809 19:30:34.378930 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:34.379060 31975 external_mini_cluster.cc:1428] Setting key ad861fe7f2d59a5d73560dc42c32515e
I20260809 19:30:34.379871 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:30:34.382071 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.381103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.31.57.254:44627)
0809 19:30:34.381159 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.381161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.381270 (+ 109us) server_negotiation.cc:407] Connection header received
0809 19:30:34.381327 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.381328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.381332 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.381384 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.381942 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.381942 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.381943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.381945 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.381954 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.381957 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.381992 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.382021 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.382031 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:30:34.383505 6912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:34.384977 6912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:34.388789 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45929 (local address 127.0.0.1:59940)
0809 19:30:34.387409 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.387444 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.387497 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.387744 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.387750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.387757 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.388262 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.388263 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.388279 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.388300 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.388627 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.388635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.388637 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.388708 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.388722 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260809 19:30:34.389178 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.31.57.193:45929)
0809 19:30:34.387539 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.387544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.387554 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:34.387582 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.387584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.387604 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.387734 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.388476 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.388479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.388480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.388482 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.388529 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.388545 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.388619 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.389076 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.389094 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":100,"threads_started":1}
I20260809 19:30:34.389417 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:34019 (local address 127.0.0.1:36302)
0809 19:30:34.387881 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.387892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.387967 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.388477 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.388478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.388481 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.389039 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.389040 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.389048 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.389061 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.389300 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.389301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.389302 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.389345 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.389357 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260809 19:30:34.389544 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.31.57.195:42147)
0809 19:30:34.387909 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.387913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.388174 (+ 261us) server_negotiation.cc:407] Connection header received
0809 19:30:34.388241 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.388245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.388254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.388333 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.389159 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.389166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.389167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.389171 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.389203 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.389245 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.389309 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.389457 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.389473 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260809 19:30:34.389613 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.31.57.194:34019)
0809 19:30:34.387675 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.387679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.388036 (+ 357us) server_negotiation.cc:407] Connection header received
0809 19:30:34.388069 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.388071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.388082 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.388165 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.389143 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.389149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.389150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.389153 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.389186 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.389208 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.389262 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.389524 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.389541 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":155,"threads_started":1}
I20260809 19:30:34.389691 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.387454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:42147 (local address 127.0.0.1:42262)
0809 19:30:34.388108 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.388129 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.388171 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.388570 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.388571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.388573 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.389064 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.389065 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.389071 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.389081 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.389336 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.389337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.389338 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.389383 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.389396 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":78,"threads_started":1}
I20260809 19:30:34.389855 7125 tablet_service.cc:1467] Processing CreateTablet for tablet 03630aa574ba492ead8622c82a65f7dd (DEFAULT_TABLE table=test-table [id=608f85278bf544acb39dc93a11ea9e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:34.390218 7125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03630aa574ba492ead8622c82a65f7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.390465 7558 tablet_service.cc:1467] Processing CreateTablet for tablet 03630aa574ba492ead8622c82a65f7dd (DEFAULT_TABLE table=test-table [id=608f85278bf544acb39dc93a11ea9e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:34.390544 7342 tablet_service.cc:1467] Processing CreateTablet for tablet 03630aa574ba492ead8622c82a65f7dd (DEFAULT_TABLE table=test-table [id=608f85278bf544acb39dc93a11ea9e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:34.390776 7558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03630aa574ba492ead8622c82a65f7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.390878 7342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03630aa574ba492ead8622c82a65f7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.393338 7684 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Bootstrap starting.
I20260809 19:30:34.393887 7687 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Bootstrap starting.
I20260809 19:30:34.393888 7685 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: Bootstrap starting.
I20260809 19:30:34.394151 7684 tablet_bootstrap.cc:654] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.394519 7684 log.cc:824] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.394548 7685 tablet_bootstrap.cc:654] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.394563 7687 tablet_bootstrap.cc:654] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.394872 7687 log.cc:824] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.394898 7685 log.cc:824] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.395326 7684 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: No bootstrap required, opened a new log
I20260809 19:30:34.395421 7684 ts_tablet_manager.cc:1397] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:34.395623 7687 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: No bootstrap required, opened a new log
I20260809 19:30:34.395709 7687 ts_tablet_manager.cc:1397] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:34.397886 7684 raft_consensus.cc:348] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.398042 7684 raft_consensus.cc:374] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.398080 7684 raft_consensus.cc:729] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11cdd35040114484b978c8374cf5df13, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.398041 7687 raft_consensus.cc:348] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.398188 7687 raft_consensus.cc:374] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.398229 7687 raft_consensus.cc:729] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c55426726947fa80b402d1a4b2fa99, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.398228 7684 consensus_queue.cc:260] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.398316 7687 consensus_queue.cc:260] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.398490 7684 ts_tablet_manager.cc:1428] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:30:34.398566 7687 ts_tablet_manager.cc:1428] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:30:34.398641 7666 heartbeater.cc:503] Master 127.31.57.254:44627 was elected leader, sending a full tablet report...
I20260809 19:30:34.398682 7455 heartbeater.cc:503] Master 127.31.57.254:44627 was elected leader, sending a full tablet report...
I20260809 19:30:34.399139 7685 tablet_bootstrap.cc:492] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: No bootstrap required, opened a new log
I20260809 19:30:34.399207 7685 ts_tablet_manager.cc:1397] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 19:30:34.401170 7685 raft_consensus.cc:348] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.401314 7685 raft_consensus.cc:374] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.401363 7685 raft_consensus.cc:729] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477d27c7487744cba0fa65387dbceb7a, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.401479 7685 consensus_queue.cc:260] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.401742 7685 ts_tablet_manager.cc:1428] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:30:34.401810 7236 heartbeater.cc:503] Master 127.31.57.254:44627 was elected leader, sending a full tablet report...
I20260809 19:30:34.442219 7694 raft_consensus.cc:482] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:34.442324 7694 raft_consensus.cc:504] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.442571 7694 leader_election.cc:288] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11cdd35040114484b978c8374cf5df13 (127.31.57.194:34019), 477d27c7487744cba0fa65387dbceb7a (127.31.57.193:45929)
I20260809 19:30:34.443854 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.442743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:34019 (local address 127.31.57.195:50399)
0809 19:30:34.442845 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.442858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.442903 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.443083 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.443084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.443086 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.443473 (+ 387us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.443474 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.443480 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.443488 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.443716 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.443716 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.443718 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.443771 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.443785 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 19:30:34.443863 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.442884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50399 (local address 127.31.57.194:34019)
0809 19:30:34.442990 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.442993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.443001 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:34.443023 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.443024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.443028 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.443074 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.443598 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.443599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.443600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.443604 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.443625 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.443630 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.443695 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.443783 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.443799 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 19:30:34.443863 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.442815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:47221 (local address 127.31.57.193:45929)
0809 19:30:34.442875 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.442877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.442995 (+ 118us) server_negotiation.cc:407] Connection header received
0809 19:30:34.443023 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.443024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.443028 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.443084 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.443642 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.443644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.443644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.443648 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.443662 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.443667 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.443722 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.443790 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.443802 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:30:34.443879 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.442744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45929 (local address 127.31.57.195:47221)
0809 19:30:34.442915 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.442925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.442982 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.443206 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.443207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.443217 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:34.443569 (+ 352us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:30:34.443569 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:34.443574 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:30:34.443584 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:34.443729 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:34.443729 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:34.443730 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:34.443780 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:30:34.443798 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":11,"thread_start_us":94,"threads_started":1}
I20260809 19:30:34.444164 7160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03630aa574ba492ead8622c82a65f7dd" candidate_uuid: "e2c55426726947fa80b402d1a4b2fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d27c7487744cba0fa65387dbceb7a" is_pre_election: true
I20260809 19:30:34.444162 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03630aa574ba492ead8622c82a65f7dd" candidate_uuid: "e2c55426726947fa80b402d1a4b2fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11cdd35040114484b978c8374cf5df13" is_pre_election: true
I20260809 19:30:34.444294 7160 raft_consensus.cc:2454] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c55426726947fa80b402d1a4b2fa99 in term 0.
I20260809 19:30:34.444308 7371 raft_consensus.cc:2454] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c55426726947fa80b402d1a4b2fa99 in term 0.
I20260809 19:30:34.444572 7480 leader_election.cc:302] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d27c7487744cba0fa65387dbceb7a, e2c55426726947fa80b402d1a4b2fa99; no voters:
I20260809 19:30:34.444728 7694 raft_consensus.cc:2781] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:34.444839 7694 raft_consensus.cc:482] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:34.444873 7694 raft_consensus.cc:3037] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.445533 7694 raft_consensus.cc:504] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.445667 7694 leader_election.cc:288] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [CANDIDATE]: Term 1 election: Requested vote from peers 11cdd35040114484b978c8374cf5df13 (127.31.57.194:34019), 477d27c7487744cba0fa65387dbceb7a (127.31.57.193:45929)
I20260809 19:30:34.445798 7371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03630aa574ba492ead8622c82a65f7dd" candidate_uuid: "e2c55426726947fa80b402d1a4b2fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11cdd35040114484b978c8374cf5df13"
I20260809 19:30:34.445825 7160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03630aa574ba492ead8622c82a65f7dd" candidate_uuid: "e2c55426726947fa80b402d1a4b2fa99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477d27c7487744cba0fa65387dbceb7a"
I20260809 19:30:34.445873 7371 raft_consensus.cc:3037] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.445879 7160 raft_consensus.cc:3037] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.446687 7371 raft_consensus.cc:2454] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c55426726947fa80b402d1a4b2fa99 in term 1.
I20260809 19:30:34.446687 7160 raft_consensus.cc:2454] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c55426726947fa80b402d1a4b2fa99 in term 1.
I20260809 19:30:34.446883 7480 leader_election.cc:302] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477d27c7487744cba0fa65387dbceb7a, e2c55426726947fa80b402d1a4b2fa99; no voters:
I20260809 19:30:34.446980 7694 raft_consensus.cc:2781] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:34.447144 7694 raft_consensus.cc:686] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 1 LEADER]: Becoming Leader. State: Replica: e2c55426726947fa80b402d1a4b2fa99, State: Running, Role: LEADER
I20260809 19:30:34.447242 7694 consensus_queue.cc:237] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } }
I20260809 19:30:34.448020 6912 catalog_manager.cc:5495] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c55426726947fa80b402d1a4b2fa99 (127.31.57.195). New cstate: current_term: 1 leader_uuid: "e2c55426726947fa80b402d1a4b2fa99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c55426726947fa80b402d1a4b2fa99" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:34.453022 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.452255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.31.57.195:42147)
0809 19:30:34.452287 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.452288 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.452296 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:34.452366 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.452366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.452376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.452422 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.452835 (+ 413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.452837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.452837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.452840 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.452851 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.452854 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.452906 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.452954 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.452966 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:30:34.454506 7371 raft_consensus.cc:1261] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 1 FOLLOWER]: Refusing update from remote peer e2c55426726947fa80b402d1a4b2fa99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:34.454506 7160 raft_consensus.cc:1261] T 03630aa574ba492ead8622c82a65f7dd P 477d27c7487744cba0fa65387dbceb7a [term 1 FOLLOWER]: Refusing update from remote peer e2c55426726947fa80b402d1a4b2fa99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:34.454779 7694 consensus_queue.cc:1035] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477d27c7487744cba0fa65387dbceb7a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:34.454892 7698 consensus_queue.cc:1035] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11cdd35040114484b978c8374cf5df13" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:34.457208 7704 mvcc.cc:205] Tried to move back new op lower bound from 7316700505923567616 to 7316700505896722432. Current Snapshot: MvccSnapshot[applied={T|T < 7316700505923567616}]
I20260809 19:30:34.457849 7703 mvcc.cc:205] Tried to move back new op lower bound from 7316700505923567616 to 7316700505896722432. Current Snapshot: MvccSnapshot[applied={T|T < 7316700505923567616}]
I20260809 19:30:34.459628 6912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:34.459695 6912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:34.460368 6912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a8413dd5d64dc0bf1374970a0adc06: Sending DeleteTablet for 3 replicas of tablet 03630aa574ba492ead8622c82a65f7dd
I20260809 19:30:34.460721 7558 tablet_service.cc:1514] Processing DeleteTablet for tablet 03630aa574ba492ead8622c82a65f7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:34 UTC) from {username='slave'} at 127.0.0.1:42262
I20260809 19:30:34.460738 7125 tablet_service.cc:1514] Processing DeleteTablet for tablet 03630aa574ba492ead8622c82a65f7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:34 UTC) from {username='slave'} at 127.0.0.1:59940
I20260809 19:30:34.460961 7714 tablet_replica.cc:330] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: stopping tablet replica
I20260809 19:30:34.460984 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 7027
I20260809 19:30:34.460995 7342 tablet_service.cc:1514] Processing DeleteTablet for tablet 03630aa574ba492ead8622c82a65f7dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:34 UTC) from {username='slave'} at 127.0.0.1:36302
I20260809 19:30:34.461150 7714 raft_consensus.cc:2229] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:34.461315 7714 raft_consensus.cc:2258] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:34.463620 7718 tablet_replica.cc:330] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: stopping tablet replica
I20260809 19:30:34.463726 7718 raft_consensus.cc:2229] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:34.463838 7718 raft_consensus.cc:2258] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:34.463950 7714 ts_tablet_manager.cc:1905] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:34.465490 7718 ts_tablet_manager.cc:1905] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:34.467877 7714 ts_tablet_manager.cc:1918] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:34.468007 7714 log.cc:1196] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/03630aa574ba492ead8622c82a65f7dd
I20260809 19:30:34.468436 7714 ts_tablet_manager.cc:1939] T 03630aa574ba492ead8622c82a65f7dd P e2c55426726947fa80b402d1a4b2fa99: Deleting consensus metadata
I20260809 19:30:34.470118 7718 ts_tablet_manager.cc:1918] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:34.470212 7718 log.cc:1196] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/03630aa574ba492ead8622c82a65f7dd
I20260809 19:30:34.470451 7718 ts_tablet_manager.cc:1939] T 03630aa574ba492ead8622c82a65f7dd P 11cdd35040114484b978c8374cf5df13: Deleting consensus metadata
I20260809 19:30:34.471908 6895 catalog_manager.cc:4841] TS e2c55426726947fa80b402d1a4b2fa99 (127.31.57.195:42147): tablet 03630aa574ba492ead8622c82a65f7dd (table test-table [id=608f85278bf544acb39dc93a11ea9e96]) successfully deleted
I20260809 19:30:34.472004 6895 catalog_manager.cc:4841] TS 11cdd35040114484b978c8374cf5df13 (127.31.57.194:34019): tablet 03630aa574ba492ead8622c82a65f7dd (table test-table [id=608f85278bf544acb39dc93a11ea9e96]) successfully deleted
W20260809 19:30:34.477820 6889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:45929 (error 108)
I20260809 19:30:34.478053 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 7243
I20260809 19:30:34.481462 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.481324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45929 (local address 127.0.0.1:59952)
0809 19:30:34.481379 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.481425 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:45929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:30:34.481572 6889 catalog_manager.cc:4584] TS 477d27c7487744cba0fa65387dbceb7a (127.31.57.193:45929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03630aa574ba492ead8622c82a65f7dd: Network error: Client connection negotiation failed: client connection to 127.31.57.193:45929: connect: Connection refused (error 111)
I20260809 19:30:34.484597 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 7459
I20260809 19:30:34.493996 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 6863
2026-08-09T19:30:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (612 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 19:30:34.516395 31975 mini_oidc.cc:152] Starting JWKS server
I20260809 19:30:34.517606 31975 webserver.cc:460] Webserver started at https://127.0.0.1:33723/ using document root <none> and password file <none>
I20260809 19:30:34.517729 31975 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:30:34.517912 31975 webserver.cc:460] Webserver started at http://127.0.0.1:38455/ using document root <none> and password file <none>
2026-08-09T19:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:34Z Disabled control of system clock
I20260809 19:30:34.525911 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39651
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39651
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33723/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/testchainca.cert with env {}
W20260809 19:30:34.605232 7732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:34.605417 7732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:34.605438 7732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:34.606506 7732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:34.606544 7732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:34.606556 7732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:34.606597 7732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:34.606622 7732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:30:34.606637 7732 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:30:34.606660 7732 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33723/jwks.json
I20260809 19:30:34.608170 7732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33723/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:34.608413 7732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:34.608631 7732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:34.613107 7741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.613130 7738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.613320 7748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.613445 7732 server_base.cc:1034] running on GCE node
I20260809 19:30:34.613672 7732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:34.613992 7732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:34.615159 7732 hybrid_clock.cc:648] HybridClock initialized: now 1786303834615134 us; error 39 us; skew 500 ppm
I20260809 19:30:34.616338 7732 webserver.cc:460] Webserver started at http://127.31.57.254:37955/ using document root <none> and password file <none>
I20260809 19:30:34.616582 7732 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:34.616652 7732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:34.616797 7732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:34.617697 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "12c9acc138424397a6132c0b57749b56"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "7fbb7ac472874a6d19ac4306f97f33b5"
server_key_iv: "b8ce4e99d98dc4b12b455e89184aec5c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.618041 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "12c9acc138424397a6132c0b57749b56"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "7fbb7ac472874a6d19ac4306f97f33b5"
server_key_iv: "b8ce4e99d98dc4b12b455e89184aec5c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.619246 7732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:34.620024 7757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.620191 7732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.620302 7732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "12c9acc138424397a6132c0b57749b56"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "7fbb7ac472874a6d19ac4306f97f33b5"
server_key_iv: "b8ce4e99d98dc4b12b455e89184aec5c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.620385 7732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.629842 7732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.630110 7732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.630246 7732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.634001 7732 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:39651
I20260809 19:30:34.634029 7848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:39651 every 8 connection(s)
I20260809 19:30:34.634351 7732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:34.635238 7849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.637161 7849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56: Bootstrap starting.
I20260809 19:30:34.637705 7849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.637977 7849 log.cc:824] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.638566 7849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56: No bootstrap required, opened a new log
I20260809 19:30:34.640071 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 7732
I20260809 19:30:34.640055 7849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } }
I20260809 19:30:34.640226 7849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.640280 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:34.640246 7849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c9acc138424397a6132c0b57749b56, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.640457 31975 external_mini_cluster.cc:1428] Setting key 559150ee58ad60473386692cd355199f
I20260809 19:30:34.640440 7849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } }
I20260809 19:30:34.640512 7849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:34.640563 7849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:34.640609 7849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.641175 7849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } }
I20260809 19:30:34.641255 7849 leader_election.cc:302] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12c9acc138424397a6132c0b57749b56; no voters:
I20260809 19:30:34.641477 7849 leader_election.cc:288] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:34.641515 7860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:34.641661 7860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [term 1 LEADER]: Becoming Leader. State: Replica: 12c9acc138424397a6132c0b57749b56, State: Running, Role: LEADER
I20260809 19:30:34.641721 7849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:34.641742 7860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } }
I20260809 19:30:34.642071 7860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12c9acc138424397a6132c0b57749b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } } }
I20260809 19:30:34.642163 7860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.642310 7860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12c9acc138424397a6132c0b57749b56. Latest consensus state: current_term: 1 leader_uuid: "12c9acc138424397a6132c0b57749b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c9acc138424397a6132c0b57749b56" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39651 } } }
I20260809 19:30:34.642365 7860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.642480 7876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:34.643029 7876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:34.646720 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.641036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.31.57.254:39651)
0809 19:30:34.641267 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.641275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.641314 (+ 39us) server_negotiation.cc:407] Connection header received
0809 19:30:34.641452 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.641471 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.641540 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.641683 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.641924 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.643682 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.644771 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.645374 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.645498 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.646067 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.646071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.646071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.646257 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.646306 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.646319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.646388 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.646447 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.646521 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260809 19:30:34.647581 7876 catalog_manager.cc:1347] Generated new cluster ID: 6437f67cfbaf46fbbdc11f2e1d23b016
I20260809 19:30:34.647629 7876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:34.647640 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:34.660655 7876 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:34.661167 7876 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:34.671931 7876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12c9acc138424397a6132c0b57749b56: Generated new TSK 0
I20260809 19:30:34.672091 7876 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:30:34.695650 7905 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 19:30:34.695786 7905 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:30:34.696000 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.682024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.31.57.254:39651)
0809 19:30:34.682064 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.682066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.682154 (+ 88us) server_negotiation.cc:407] Connection header received
0809 19:30:34.682251 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.682251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.682260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.682314 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:30:34.682472 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.682887 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.683978 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.684151 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.684203 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.684443 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:30:34.695794 (+ 11351us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:30:34.695952 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 19:30:34.696062 7858 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 19:30:34.696380 7903 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:34.681895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39651 (local address 127.0.0.1:47990)
0809 19:30:34.682080 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.682090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.682154 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.682338 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.682339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.682341 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:30:34.682406 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:34.682408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.682962 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.682963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:34.683897 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:34.683901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.684227 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.684229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:34.684303 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.684325 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:30:34.695965 (+ 11640us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:30:34.696070 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.57.254:39651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.57.254:39651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 19:30:34.696787 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 7732
2026-08-09T19:30:34Z chronyd exiting
I20260809 19:30:34.741585 31975 test_util.cc:175] -----------------------------------------------
I20260809 19:30:34.741649 31975 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303811347120-31975-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 19:30:34.742380 31975 mini_oidc.cc:152] Starting JWKS server
I20260809 19:30:34.743031 31975 webserver.cc:460] Webserver started at https://127.0.0.1:43081/ using document root <none> and password file <none>
I20260809 19:30:34.743099 31975 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:30:34.743266 31975 webserver.cc:460] Webserver started at http://127.0.0.1:35779/ using document root <none> and password file <none>
2026-08-09T19:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:34Z Disabled control of system clock
I20260809 19:30:34.751479 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39235
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:36885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39235
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43081/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/testchainca.cert with env {}
W20260809 19:30:34.827587 7918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:34.827761 7918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:34.827783 7918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:34.828954 7918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:34.828994 7918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:34.829007 7918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:34.829058 7918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:34.829084 7918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:30:34.829116 7918 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:30:34.829161 7918 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43081/jwks.json
I20260809 19:30:34.830600 7918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43081/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:34.830838 7918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:34.831054 7918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:34.835635 7925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.835654 7918 server_base.cc:1034] running on GCE node
W20260809 19:30:34.835635 7929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:34.835654 7924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:34.836050 7918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:34.836346 7918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:34.837507 7918 hybrid_clock.cc:648] HybridClock initialized: now 1786303834837500 us; error 35 us; skew 500 ppm
I20260809 19:30:34.838559 7918 webserver.cc:460] Webserver started at http://127.31.57.254:33757/ using document root <none> and password file <none>
I20260809 19:30:34.838795 7918 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:34.838863 7918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:34.839000 7918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:34.839972 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "f3227eb47f13400d988c5e56be7d36f4"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "e5b29704579801120b20857949ed6c01"
server_key_iv: "45d323a71c882c2568b2f72151a4255c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.840308 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "f3227eb47f13400d988c5e56be7d36f4"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "e5b29704579801120b20857949ed6c01"
server_key_iv: "45d323a71c882c2568b2f72151a4255c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.841526 7918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:34.842216 7939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:34.842384 7918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:34.842483 7918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "f3227eb47f13400d988c5e56be7d36f4"
format_stamp: "Formatted at 2026-08-09 19:30:34 on dist-test-slave-x4qh"
server_key: "e5b29704579801120b20857949ed6c01"
server_key_iv: "45d323a71c882c2568b2f72151a4255c"
server_key_version: "encryptionkey@0"
I20260809 19:30:34.842573 7918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:34.866595 7918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:34.866855 7918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:34.866997 7918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:34.870708 7918 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:39235
I20260809 19:30:34.870745 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:39235 every 8 connection(s)
I20260809 19:30:34.871095 7918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:34.871767 8030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:34.874342 8030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4: Bootstrap starting.
I20260809 19:30:34.874892 8030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:34.875198 8030 log.cc:824] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:34.875653 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 7918
I20260809 19:30:34.875753 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:34.875942 31975 external_mini_cluster.cc:1428] Setting key cf98bd2e7db22b38210aaf5363c7462b
I20260809 19:30:34.875960 8030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4: No bootstrap required, opened a new log
I20260809 19:30:34.878259 8030 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } }
I20260809 19:30:34.878420 8030 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:34.878461 8030 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3227eb47f13400d988c5e56be7d36f4, State: Initialized, Role: FOLLOWER
I20260809 19:30:34.878573 8030 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } }
I20260809 19:30:34.878641 8030 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:34.878679 8030 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:34.878722 8030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:34.879417 8030 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } }
I20260809 19:30:34.879539 8030 leader_election.cc:302] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3227eb47f13400d988c5e56be7d36f4; no voters:
I20260809 19:30:34.879755 8030 leader_election.cc:288] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:34.879784 8038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:34.879909 8038 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [term 1 LEADER]: Becoming Leader. State: Replica: f3227eb47f13400d988c5e56be7d36f4, State: Running, Role: LEADER
I20260809 19:30:34.879997 8038 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } }
I20260809 19:30:34.880024 8030 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:34.880975 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3227eb47f13400d988c5e56be7d36f4. Latest consensus state: current_term: 1 leader_uuid: "f3227eb47f13400d988c5e56be7d36f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } } }
I20260809 19:30:34.881062 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.881115 8040 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3227eb47f13400d988c5e56be7d36f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3227eb47f13400d988c5e56be7d36f4" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39235 } } }
I20260809 19:30:34.881196 8040 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:34.881562 8054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:34.882179 8054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:34.882215 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.876432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.31.57.254:39235)
0809 19:30:34.876621 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.876627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.876648 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:30:34.877177 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.877182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.877230 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.877331 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:34.877540 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.878292 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.879444 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.880070 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.880510 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.880891 (+ 381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:34.881506 (+ 615us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:34.881506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:34.881746 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:34.881821 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:34.881838 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:34.881904 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:34.881991 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:30:34.882064 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260809 19:30:34.883019 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:34.883606 8054 catalog_manager.cc:1347] Generated new cluster ID: aea9232170a34847b72adff1825d49e5
I20260809 19:30:34.883661 8054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:34.896615 8054 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:34.897055 8054 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:34.903542 8054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3227eb47f13400d988c5e56be7d36f4: Generated new TSK 0
I20260809 19:30:34.903707 8054 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:30:34.931648 8079 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 19:30:34.931739 8079 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:30:34.931975 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:34.911689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.31.57.254:39235)
0809 19:30:34.911720 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:34.911722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:34.911817 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:30:34.911883 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:34.911883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:34.911892 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:34.911943 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:30:34.912043 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.912462 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.913450 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.913630 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.913675 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.913857 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:30:34.931805 (+ 17948us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:30:34.931930 (+ 125us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:30:34.932039 8037 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 19:30:34.932073 8077 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:34.911641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39235 (local address 127.0.0.1:60242)
0809 19:30:34.911761 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:34.911769 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:34.911823 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:34.911948 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:34.911949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:34.911950 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:30:34.911995 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:34.911996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.912538 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.912539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:34.913375 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:34.913377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:34.913706 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:34.913707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:34.913767 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:34.913772 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:30:34.931904 (+ 18132us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:30:34.931992 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.57.254:39235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43081/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260809 19:30:34.932411 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 7918
2026-08-09T19:30:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 19:30:34.983769 31975 mini_oidc.cc:152] Starting JWKS server
I20260809 19:30:34.984408 31975 webserver.cc:460] Webserver started at https://127.0.0.1:39555/ using document root <none> and password file <none>
I20260809 19:30:34.984447 31975 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:30:34.984553 31975 webserver.cc:460] Webserver started at http://127.0.0.1:41835/ using document root <none> and password file <none>
2026-08-09T19:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:34Z Disabled control of system clock
I20260809 19:30:34.993024 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46685
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46685
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39555/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:30:35.070214 8089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:35.070420 8089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:35.070458 8089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:35.071694 8089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:35.071759 8089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:35.071782 8089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:35.071807 8089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:35.071846 8089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:30:35.071894 8089 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:30:35.071933 8089 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39555/jwks.json
I20260809 19:30:35.073175 8089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39555/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:35.073453 8089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:35.073770 8089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:35.078374 8099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:35.078337 8097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:35.078352 8095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:35.078545 8089 server_base.cc:1034] running on GCE node
I20260809 19:30:35.078802 8089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:35.079070 8089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:35.080220 8089 hybrid_clock.cc:648] HybridClock initialized: now 1786303835080204 us; error 28 us; skew 500 ppm
I20260809 19:30:35.081477 8089 webserver.cc:460] Webserver started at http://127.31.57.254:42585/ using document root <none> and password file <none>
I20260809 19:30:35.081717 8089 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:35.081758 8089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:35.081844 8089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:35.082749 8089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "4de89387c7c54030bf115a2999b3a0ca"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "13b99e427238280d4ae446b8af0d4d3e"
server_key_iv: "10061b7d438ded2aaec3695f367124f1"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.083016 8089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "4de89387c7c54030bf115a2999b3a0ca"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "13b99e427238280d4ae446b8af0d4d3e"
server_key_iv: "10061b7d438ded2aaec3695f367124f1"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.084380 8089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:35.085146 8112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:35.085336 8089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:35.085413 8089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "4de89387c7c54030bf115a2999b3a0ca"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "13b99e427238280d4ae446b8af0d4d3e"
server_key_iv: "10061b7d438ded2aaec3695f367124f1"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.085492 8089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:35.098026 8089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:35.098264 8089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:35.098366 8089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:35.102322 8089 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:46685
I20260809 19:30:35.102349 8207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:46685 every 8 connection(s)
I20260809 19:30:35.102658 8089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:35.103344 8208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:35.105749 8208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca: Bootstrap starting.
I20260809 19:30:35.106212 8208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:35.106439 8208 log.cc:824] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:35.107077 8208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca: No bootstrap required, opened a new log
I20260809 19:30:35.107187 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 8089
I20260809 19:30:35.107270 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:35.107422 31975 external_mini_cluster.cc:1428] Setting key 3993b4685812022760ce6c9285276714
I20260809 19:30:35.108899 8208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } }
I20260809 19:30:35.109042 8208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:35.109077 8208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de89387c7c54030bf115a2999b3a0ca, State: Initialized, Role: FOLLOWER
I20260809 19:30:35.109198 8208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } }
I20260809 19:30:35.109282 8208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:35.109340 8208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:35.109388 8208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:35.109908 8208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } }
I20260809 19:30:35.109989 8208 leader_election.cc:302] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4de89387c7c54030bf115a2999b3a0ca; no voters:
I20260809 19:30:35.110157 8208 leader_election.cc:288] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:35.110179 8213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:35.110301 8213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 4de89387c7c54030bf115a2999b3a0ca, State: Running, Role: LEADER
I20260809 19:30:35.110379 8213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } }
I20260809 19:30:35.110450 8208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:35.110687 8215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4de89387c7c54030bf115a2999b3a0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } } }
I20260809 19:30:35.110913 8215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:35.110842 8216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4de89387c7c54030bf115a2999b3a0ca. Latest consensus state: current_term: 1 leader_uuid: "4de89387c7c54030bf115a2999b3a0ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de89387c7c54030bf115a2999b3a0ca" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46685 } } }
I20260809 19:30:35.111347 8216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:35.111536 8226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:35.111966 8226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:35.113987 8226 catalog_manager.cc:1347] Generated new cluster ID: ed00a183848142c3b2d97d8ed04fb1dd
I20260809 19:30:35.114013 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:35.107841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.31.57.254:46685)
0809 19:30:35.108046 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:35.108052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:35.108074 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:30:35.108887 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:35.108892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:35.108939 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:35.109053 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:35.109243 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.110871 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.111949 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.112507 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.112629 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.113331 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:35.113335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:35.113335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:35.113504 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:35.113535 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:35.113545 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:35.113621 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:35.113733 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:30:35.113819 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260809 19:30:35.114039 8226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:35.115026 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:35.124182 8226 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:35.124825 8226 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:35.131893 8226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4de89387c7c54030bf115a2999b3a0ca: Generated new TSK 0
I20260809 19:30:35.132047 8226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:30:35.140264 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:35.136557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.31.57.254:46685)
0809 19:30:35.136607 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:35.136609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:35.136667 (+ 58us) server_negotiation.cc:407] Connection header received
0809 19:30:35.136738 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:35.136738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:35.136746 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:35.136801 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:30:35.136951 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.137705 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.138767 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.138940 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.138995 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.139175 (+ 180us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:30:35.140083 (+ 908us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39555/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:30:35.140206 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39555/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:30:35.140328 8212 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39555/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 19:30:35.140395 8250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:35.136453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46685 (local address 127.0.0.1:60852)
0809 19:30:35.136605 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:35.136616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:35.136676 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:35.136821 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:35.136822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:35.136824 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:30:35.136873 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:35.136875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.137781 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.137782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:35.138689 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:35.138692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.139027 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.139028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:35.139087 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.139092 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:30:35.140193 (+ 1101us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39555/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:30:35.140286 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.57.254:46685: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39555/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 19:30:35.140866 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 8089
E20260809 19:30:35.158241 8252 webserver.cc:533] Webserver: sslize failed:
E20260809 19:30:35.158325 8252 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T19:30:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T19:30:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:35Z Disabled control of system clock
I20260809 19:30:35.195241 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33831
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33831
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/testchainca.cert with env {}
W20260809 19:30:35.269402 8264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:35.269613 8264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:35.269665 8264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:35.270844 8264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:35.270910 8264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:35.270931 8264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:35.270951 8264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:35.270978 8264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:30:35.271003 8264 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:30:35.271028 8264 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 19:30:35.272399 8264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:35.272688 8264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:35.272898 8264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:35.277455 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:35.277462 8290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:35.277462 8278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:35.277784 8264 server_base.cc:1034] running on GCE node
I20260809 19:30:35.278017 8264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:35.278333 8264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:35.279448 8264 hybrid_clock.cc:648] HybridClock initialized: now 1786303835279434 us; error 27 us; skew 500 ppm
I20260809 19:30:35.280699 8264 webserver.cc:460] Webserver started at http://127.31.57.254:39609/ using document root <none> and password file <none>
I20260809 19:30:35.280944 8264 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:35.280990 8264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:35.281128 8264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:35.282068 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "92b7accd3bdf4ce3b0b4817e7151c825"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "b14714737092076b055ab56a52bdf23e"
server_key_iv: "c049c0a9034fed4f34508a5edb147c2e"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.282413 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "92b7accd3bdf4ce3b0b4817e7151c825"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "b14714737092076b055ab56a52bdf23e"
server_key_iv: "c049c0a9034fed4f34508a5edb147c2e"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.283663 8264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:30:35.284327 8299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:35.284518 8264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:35.284642 8264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "92b7accd3bdf4ce3b0b4817e7151c825"
format_stamp: "Formatted at 2026-08-09 19:30:35 on dist-test-slave-x4qh"
server_key: "b14714737092076b055ab56a52bdf23e"
server_key_iv: "c049c0a9034fed4f34508a5edb147c2e"
server_key_version: "encryptionkey@0"
I20260809 19:30:35.284716 8264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:35.294256 8264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:35.294519 8264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:35.294657 8264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:35.298609 8264 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:33831
I20260809 19:30:35.298640 8393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:33831 every 8 connection(s)
I20260809 19:30:35.298954 8264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:35.299382 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 8264
I20260809 19:30:35.299499 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:35.299683 31975 external_mini_cluster.cc:1428] Setting key 9b6d3e595ab82d412f709f407897d814
I20260809 19:30:35.300119 8394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:35.303050 8394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825: Bootstrap starting.
I20260809 19:30:35.303828 8394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:35.304178 8394 log.cc:824] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:35.304921 8394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825: No bootstrap required, opened a new log
I20260809 19:30:35.305008 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:35.300173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.31.57.254:33831)
0809 19:30:35.300395 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:35.300401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:35.300427 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:30:35.300803 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:35.300809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:35.300860 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:35.300963 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:35.301200 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.302026 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.303092 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.303613 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.303712 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.304401 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:35.304405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:35.304405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:30:35.304552 (+ 147us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:35.304583 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:30:35.304593 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:35.304668 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:35.304749 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:30:35.304823 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260809 19:30:35.305917 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:35.307102 8394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } }
I20260809 19:30:35.307217 8394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:35.307237 8394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b7accd3bdf4ce3b0b4817e7151c825, State: Initialized, Role: FOLLOWER
I20260809 19:30:35.307384 8394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } }
I20260809 19:30:35.307461 8394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:35.307508 8394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:35.307556 8394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:35.308076 8394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } }
I20260809 19:30:35.308182 8394 leader_election.cc:302] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b7accd3bdf4ce3b0b4817e7151c825; no voters:
I20260809 19:30:35.308333 8394 leader_election.cc:288] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:35.308363 8401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:35.308465 8401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [term 1 LEADER]: Becoming Leader. State: Replica: 92b7accd3bdf4ce3b0b4817e7151c825, State: Running, Role: LEADER
I20260809 19:30:35.308538 8401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } }
I20260809 19:30:35.308566 8394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:35.308920 8404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b7accd3bdf4ce3b0b4817e7151c825. Latest consensus state: current_term: 1 leader_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } } }
I20260809 19:30:35.308920 8402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b7accd3bdf4ce3b0b4817e7151c825" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33831 } } }
I20260809 19:30:35.309083 8402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:35.309266 8404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:35.309459 8414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:35.309957 8414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:35.311303 8414 catalog_manager.cc:1347] Generated new cluster ID: a063eb9b357843849788b6ec19be97d9
I20260809 19:30:35.311350 8414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:35.337705 8414 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:35.338191 8414 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:35.347411 8414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92b7accd3bdf4ce3b0b4817e7151c825: Generated new TSK 0
I20260809 19:30:35.347596 8414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:30:36.224606 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:35.360705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.31.57.254:33831)
0809 19:30:35.360781 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:35.360783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:35.360937 (+ 154us) server_negotiation.cc:407] Connection header received
0809 19:30:35.360999 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:35.360999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:35.361012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:35.361077 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:30:35.361229 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.361722 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.362766 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.362968 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.363021 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.363215 (+ 194us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:30:36.224359 (+861144us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:30:36.224545 (+ 186us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50374: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 19:30:36.224699 8397 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50374: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 19:30:36.224714 8443 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:35.360673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33831 (local address 127.0.0.1:50374)
0809 19:30:35.360856 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:35.360869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:35.360937 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:35.361093 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:35.361094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:35.361096 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:30:35.361159 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:35.361162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.361806 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.361808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:35.362677 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:35.362680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:35.363052 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:35.363053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:35.363108 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:35.363114 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:30:36.224501 (+861387us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:30:36.224637 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.57.254:33831: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260809 19:30:36.225097 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 8264
2026-08-09T19:30:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1065 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (84 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (82 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:36 dist-test-slave-x4qh krb5kdc[8465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:36 dist-test-slave-x4qh krb5kdc[8465](info): set up 2 sockets
Aug 09 19:30:36 dist-test-slave-x4qh krb5kdc[8465](info): commencing operation
krb5kdc: starting...
W20260809 19:30:37.570149 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:37 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:37.679628 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41719
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:37.754338 8489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:37.754518 8489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:37.754539 8489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:37.755580 8489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:37.755617 8489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:37.755630 8489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:37.755640 8489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:37.755685 8489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:37.756922 8489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:37.757108 8489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:37.757315 8489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:37.761924 8498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:37.761909 8500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:37.761909 8504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:37.762036 8489 server_base.cc:1034] running on GCE node
I20260809 19:30:37.762393 8489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:37.762681 8489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:37.763841 8489 hybrid_clock.cc:648] HybridClock initialized: now 1786303837763833 us; error 39 us; skew 500 ppm
Aug 09 19:30:37 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303837, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:37.766175 8489 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:37.766647 8489 webserver.cc:460] Webserver started at http://127.31.57.254:36875/ using document root <none> and password file <none>
I20260809 19:30:37.766817 8489 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:37.766856 8489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:37.766937 8489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:37.767900 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "30befc2221ff46d99f21820dfd9fff8b"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "3df81fe0f3e1cdc0b0385ebf71de3089"
server_key_iv: "c4580e653b040f361b26c78b71463f91"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.768173 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "30befc2221ff46d99f21820dfd9fff8b"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "3df81fe0f3e1cdc0b0385ebf71de3089"
server_key_iv: "c4580e653b040f361b26c78b71463f91"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.769349 8489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:37.770084 8517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:37.770282 8489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:37.770385 8489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "30befc2221ff46d99f21820dfd9fff8b"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "3df81fe0f3e1cdc0b0385ebf71de3089"
server_key_iv: "c4580e653b040f361b26c78b71463f91"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.770460 8489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:37.791237 8489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:37.792862 8489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:37.792999 8489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:37.796865 8489 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41719
I20260809 19:30:37.796932 8594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41719 every 8 connection(s)
I20260809 19:30:37.797227 8489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:37.798014 8597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:37.800071 8597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b: Bootstrap starting.
I20260809 19:30:37.800637 8597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:37.800940 8597 log.cc:824] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:37.801617 8597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b: No bootstrap required, opened a new log
I20260809 19:30:37.803025 8597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } }
I20260809 19:30:37.803180 8597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:37.803228 8597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30befc2221ff46d99f21820dfd9fff8b, State: Initialized, Role: FOLLOWER
I20260809 19:30:37.803388 8597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } }
I20260809 19:30:37.803537 8597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:37.803580 8597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:37.803632 8597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:37.803818 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 8489
I20260809 19:30:37.803901 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:37.804080 31975 external_mini_cluster.cc:1428] Setting key 17d235cad9cbe7ea9a1274955bf41aa3
I20260809 19:30:37.804270 8597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } }
I20260809 19:30:37.804414 8597 leader_election.cc:302] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30befc2221ff46d99f21820dfd9fff8b; no voters:
I20260809 19:30:37.804595 8597 leader_election.cc:288] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:37.804765 8604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:37.804947 8597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:37.804955 8604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [term 1 LEADER]: Becoming Leader. State: Replica: 30befc2221ff46d99f21820dfd9fff8b, State: Running, Role: LEADER
I20260809 19:30:37.805151 8604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } }
I20260809 19:30:37.806109 8610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30befc2221ff46d99f21820dfd9fff8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } } }
I20260809 19:30:37.806206 8610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:37.806540 8611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30befc2221ff46d99f21820dfd9fff8b. Latest consensus state: current_term: 1 leader_uuid: "30befc2221ff46d99f21820dfd9fff8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30befc2221ff46d99f21820dfd9fff8b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41719 } } }
I20260809 19:30:37.806430 8635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:37.806663 8611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:37.807065 8635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:37.808615 8635 catalog_manager.cc:1347] Generated new cluster ID: 8415218773184cc4965f578c916c56dd
I20260809 19:30:37.808705 8635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:30:37 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:37.813860 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:37.804516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.31.57.254:41719)
0809 19:30:37.804805 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:37.804813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:37.804842 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:30:37.804982 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:37.804989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:37.805049 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:37.805188 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:37.805958 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.807238 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.808072 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.808574 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.808674 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:37.810772 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:37.810775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:37.810775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:37.810785 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:37.812341 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:37.812700 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:37.812702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:37.812703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:37.812804 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:37.813006 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:37.813007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:37.813007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:37.813229 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:37.813422 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:37.813503 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:30:37.813725 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":91,"threads_started":1}
I20260809 19:30:37.822826 8635 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:37.823506 8635 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:37.838917 8635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30befc2221ff46d99f21820dfd9fff8b: Generated new TSK 0
I20260809 19:30:37.839083 8635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:37.866153 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:37.941872 8647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:37.942047 8647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:37.942068 8647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:37.943184 8647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:37.943225 8647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:37.943250 8647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:37.944748 8647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:37.944981 8647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:37.945197 8647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:37.949729 8653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:37.949748 8654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:37.949748 8657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:37.949910 8647 server_base.cc:1034] running on GCE node
I20260809 19:30:37.950189 8647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:37.950429 8647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:37.951583 8647 hybrid_clock.cc:648] HybridClock initialized: now 1786303837951573 us; error 42 us; skew 500 ppm
Aug 09 19:30:37 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303837, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:37.953874 8647 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:37.954280 8647 webserver.cc:460] Webserver started at http://127.31.57.193:40193/ using document root <none> and password file <none>
I20260809 19:30:37.954530 8647 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:37.954612 8647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:37.954758 8647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:37.955785 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "1975d2f2a53f4739bdbdad41a4435f9e"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "eb58474e1d48f44bc1a5d2dcb4b7e263"
server_key_iv: "dbbe3b23246f6716840dc4a394770d88"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.956115 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "1975d2f2a53f4739bdbdad41a4435f9e"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "eb58474e1d48f44bc1a5d2dcb4b7e263"
server_key_iv: "dbbe3b23246f6716840dc4a394770d88"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.957345 8647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:37.958035 8667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:37.958207 8647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:37.958302 8647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "1975d2f2a53f4739bdbdad41a4435f9e"
format_stamp: "Formatted at 2026-08-09 19:30:37 on dist-test-slave-x4qh"
server_key: "eb58474e1d48f44bc1a5d2dcb4b7e263"
server_key_iv: "dbbe3b23246f6716840dc4a394770d88"
server_key_version: "encryptionkey@0"
I20260809 19:30:37.958381 8647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:37.966701 8647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:37.968446 8647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:37.968600 8647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:37.968812 8647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:37.969130 8647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:37.969200 8647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:37.969237 8647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:37.969265 8647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:37.975135 8647 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:43759
I20260809 19:30:37.975170 8847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:43759 every 8 connection(s)
I20260809 19:30:37.975526 8647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:37.980427 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 8647
I20260809 19:30:37.980552 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:37.980726 31975 external_mini_cluster.cc:1428] Setting key c1726d643762de61eb8ff8f69e9dc849
Aug 09 19:30:37 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303837, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:37.984683 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:37.976280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:36185 (local address 127.31.57.254:41719)
0809 19:30:37.976339 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:37.976341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:37.976509 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:30:37.977318 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:37.977319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:37.977331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:37.977381 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:37.977884 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.978331 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.979186 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.979357 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.979460 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:37.981624 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:37.981625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:37.981625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:37.981632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:37.983251 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:37.983694 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:37.983695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:37.983695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:37.983762 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:37.984002 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:37.984003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:37.984003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:37.984160 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:37.984234 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:37.984504 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 19:30:37.984595 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:37.984845 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:37.976257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41719 (local address 127.31.57.193:36185)
0809 19:30:37.976397 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:37.976434 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:37.977236 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:37.977428 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:37.977430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:37.977639 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:37.977808 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:37.977811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.978407 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.978407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:37.979108 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:37.979110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:37.979490 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:37.979491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:37.979599 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:37.979963 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:37.979970 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:37.981534 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:37.983338 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:37.983345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:37.983346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:37.983616 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:37.983836 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:37.983836 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:37.983837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:37.983932 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:37.984231 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:37.984235 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:37.984354 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 19:30:37.984466 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 19:30:37.984621 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260809 19:30:37.985358 8851 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41719
I20260809 19:30:37.985440 8851 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:37.985611 8851 heartbeater.cc:511] Master 127.31.57.254:41719 requested a full tablet report, sending...
I20260809 19:30:37.986127 8543 ts_manager.cc:194] Registered new tserver with Master: 1975d2f2a53f4739bdbdad41a4435f9e (127.31.57.193:43759)
I20260809 19:30:37.986704 8543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:36185
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:38.003875 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:38.083280 8858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:38.083607 8858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:38.083647 8858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:38.085661 8858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:38.085705 8858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:38.085739 8858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:38.087711 8858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:38.087993 8858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:38.088264 8858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:38.094273 8866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:38.094286 8865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:38.094427 8858 server_base.cc:1034] running on GCE node
W20260809 19:30:38.094448 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:38.094686 8858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:38.094923 8858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:38.096902 8858 hybrid_clock.cc:648] HybridClock initialized: now 1786303838096880 us; error 38 us; skew 500 ppm
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.099498 8858 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:38.099999 8858 webserver.cc:460] Webserver started at http://127.31.57.194:34069/ using document root <none> and password file <none>
I20260809 19:30:38.100245 8858 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:38.100309 8858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:38.100445 8858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:38.101943 8858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "4c9b4c3d472e4d91bcee73a4404784fe"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "c9bb476e2172913c0d0c4127f2778fdf"
server_key_iv: "e77c5068d693db6727c621de26a6b1a0"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.102367 8858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "4c9b4c3d472e4d91bcee73a4404784fe"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "c9bb476e2172913c0d0c4127f2778fdf"
server_key_iv: "e77c5068d693db6727c621de26a6b1a0"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.104235 8858 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 19:30:38.106465 8878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.106611 8858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:38.106685 8858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "4c9b4c3d472e4d91bcee73a4404784fe"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "c9bb476e2172913c0d0c4127f2778fdf"
server_key_iv: "e77c5068d693db6727c621de26a6b1a0"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.106746 8858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:38.119573 8858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:38.121672 8858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:38.121814 8858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:38.122018 8858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:38.122320 8858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:38.122351 8858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.122390 8858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:38.122414 8858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.128448 8858 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:33357
I20260809 19:30:38.128490 9056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:33357 every 8 connection(s)
I20260809 19:30:38.128748 8858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:38.136415 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.129278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:40511 (local address 127.31.57.254:41719)
0809 19:30:38.129349 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.129351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.129529 (+ 178us) server_negotiation.cc:407] Connection header received
0809 19:30:38.130329 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.130330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.130340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.130386 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.130898 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.131322 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.132183 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.132344 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.132388 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.134100 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:38.134101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:38.134101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:38.134108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:38.135459 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.135766 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.135767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.135767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.135818 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.135999 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.135999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.135999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.136082 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:38.136124 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:38.136301 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:30:38.136359 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:30:38.136516 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.129245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41719 (local address 127.31.57.194:40511)
0809 19:30:38.129403 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:38.129431 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:38.130247 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:38.130431 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:38.130434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:38.130664 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:38.130820 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:38.130824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.131434 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.131435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:38.132114 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:38.132116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.132409 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.132410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:38.132494 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.132813 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:38.132819 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:38.134017 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:38.135519 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:38.135524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:38.135525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:38.135710 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:38.135869 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:38.135870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:38.135870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:38.135951 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:38.136124 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:38.136127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:38.136211 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 19:30:38.136276 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:30:38.136370 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
I20260809 19:30:38.136979 9057 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41719
I20260809 19:30:38.137049 9057 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:38.137215 9057 heartbeater.cc:511] Master 127.31.57.254:41719 requested a full tablet report, sending...
I20260809 19:30:38.137575 8543 ts_manager.cc:194] Registered new tserver with Master: 4c9b4c3d472e4d91bcee73a4404784fe (127.31.57.194:33357)
I20260809 19:30:38.137907 8543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:40511
I20260809 19:30:38.138599 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 8858
I20260809 19:30:38.138703 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:38.138850 31975 external_mini_cluster.cc:1428] Setting key e3916d440b58bb1627266b0dd85da5f5
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:38.162000 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:42021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:38.236264 9065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:38.236433 9065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:38.236465 9065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:38.237409 9065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:38.237439 9065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:38.237461 9065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:38.238701 9065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:42021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:38.238852 9065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:38.238996 9065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:38.243592 9078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:38.243566 9080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:38.243570 9075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:38.243784 9065 server_base.cc:1034] running on GCE node
I20260809 19:30:38.244009 9065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:38.244247 9065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:38.245402 9065 hybrid_clock.cc:648] HybridClock initialized: now 1786303838245400 us; error 43 us; skew 500 ppm
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.247602 9065 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:38.247989 9065 webserver.cc:460] Webserver started at http://127.31.57.195:42113/ using document root <none> and password file <none>
I20260809 19:30:38.248203 9065 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:38.248268 9065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:38.248399 9065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:38.249356 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "157cc411f8d5405c839864d4aad1adea"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "0fa465981dc6b8b8329899a9c5cc7e9a"
server_key_iv: "50ddb7ab3998a065a234f971376c3a1c"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.249696 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "157cc411f8d5405c839864d4aad1adea"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "0fa465981dc6b8b8329899a9c5cc7e9a"
server_key_iv: "50ddb7ab3998a065a234f971376c3a1c"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.250905 9065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:30:38.251587 9090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.251746 9065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:38.251838 9065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "157cc411f8d5405c839864d4aad1adea"
format_stamp: "Formatted at 2026-08-09 19:30:38 on dist-test-slave-x4qh"
server_key: "0fa465981dc6b8b8329899a9c5cc7e9a"
server_key_iv: "50ddb7ab3998a065a234f971376c3a1c"
server_key_version: "encryptionkey@0"
I20260809 19:30:38.251914 9065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:38.260816 9065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:38.262866 9065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:38.263017 9065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:38.263216 9065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:38.263561 9065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:38.263595 9065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.263615 9065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:38.263657 9065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:38.268762 9065 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:35665
I20260809 19:30:38.268844 9262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:35665 every 8 connection(s)
I20260809 19:30:38.269110 9065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:38.276074 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 9065
I20260809 19:30:38.276186 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:38.276369 31975 external_mini_cluster.cc:1428] Setting key 258e4fb237ec929218b2b383efe654b0
I20260809 19:30:38.278049 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.269774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:45237 (local address 127.31.57.254:41719)
0809 19:30:38.269816 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.269818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.270000 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:30:38.270929 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.270931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.270941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.270997 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.271595 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.272037 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.272917 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.273080 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.273129 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.275045 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:38.275047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:38.275047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:38.275052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:38.277106 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.277426 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.277427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.277427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.277474 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.277625 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.277625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.277625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.277704 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:38.277744 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:38.277940 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 19:30:38.277994 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:38.278203 9267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.269728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41719 (local address 127.31.57.195:45237)
0809 19:30:38.269881 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:38.269915 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:38.270839 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:38.271046 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:38.271049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:38.271277 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:38.271524 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:38.271529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.272109 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.272109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:38.272827 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:38.272828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.273175 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.273176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:38.273270 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.273734 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:38.273742 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:38.274966 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:38.277173 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:38.277179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:38.277180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:38.277367 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:38.277514 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:38.277514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:38.277515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:38.277585 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:38.277745 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:38.277747 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:38.277838 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 19:30:38.277909 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:30:38.278006 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260809 19:30:38.278645 9263 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41719
I20260809 19:30:38.278736 9263 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:38.278906 9263 heartbeater.cc:511] Master 127.31.57.254:41719 requested a full tablet report, sending...
I20260809 19:30:38.279278 8543 ts_manager.cc:194] Registered new tserver with Master: 157cc411f8d5405c839864d4aad1adea (127.31.57.195:35665)
I20260809 19:30:38.279642 8543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:45237
I20260809 19:30:38.288048 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/test-admin.keytab.
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.301759 31975 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 19:30:38.303350 9277 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:38.302587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.31.57.193:43759)
0809 19:30:38.302763 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.302767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.302777 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:38.302800 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.302802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.302815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.302878 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.303261 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51190: BlockingRecv error: recv got EOF from 127.0.0.1:51190 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
W20260809 19:30:38.303403 31975 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.304301 31975 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.305197 31975 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:38.312740 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.305679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.31.57.193:43759)
0809 19:30:38.305720 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.305739 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.305748 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:38.305807 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.305808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.305818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.305885 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.306354 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.307294 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.308048 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.308380 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.308460 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.310253 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:38.310261 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:38.310261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:38.310267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:38.311616 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.311970 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.311973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.311973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.312030 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.312237 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.312238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.312239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.312365 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:38.312522 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:38.312572 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:30:38.312638 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:30:38.314477 9277 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:38.313895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.31.57.193:43759)
0809 19:30:38.313927 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.313928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.313993 (+ 65us) server_negotiation.cc:407] Connection header received
0809 19:30:38.314050 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.314051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.314058 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.314095 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.314430 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51202: BlockingRecv error: recv got EOF from 127.0.0.1:51202 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:30:38.314585 31975 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.315466 31975 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.316197 31975 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:38.322265 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.316674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.31.57.193:43759)
0809 19:30:38.316740 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.316742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.316787 (+ 45us) server_negotiation.cc:407] Connection header received
0809 19:30:38.316835 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.316836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.316843 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.316884 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.317182 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.317576 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.318159 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.318330 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.318388 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.320362 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:38.320363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:38.320364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:38.320369 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:38.321423 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.321699 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.321700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.321700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.321744 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.321897 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.321897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.321897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.321965 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:38.322022 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:38.322099 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:30:38.322150 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 19:30:38.323689 9277 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:38.323103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.31.57.193:43759)
0809 19:30:38.323168 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.323170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.323233 (+ 63us) server_negotiation.cc:407] Connection header received
0809 19:30:38.323288 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.323289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.323297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.323348 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.323649 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51218: BlockingRecv error: recv got EOF from 127.0.0.1:51218 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 19:30:38.323771 31975 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303811347120-31975-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.324610 31975 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:38.325440 31975 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[8465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:30:38.331354 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:38.325872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.31.57.193:43759)
0809 19:30:38.325927 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:38.325929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:38.325973 (+ 44us) server_negotiation.cc:407] Connection header received
0809 19:30:38.326036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:38.326037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:38.326044 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:38.326106 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:38.326434 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.326844 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.327540 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:38.327702 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:38.327769 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:38.329367 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:38.329368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:38.329368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:38.329373 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:38.330338 (+ 965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.330597 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.330598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.330598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.330640 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:38.330769 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:38.330769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:38.330769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:38.330849 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:38.330903 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:38.331235 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 19:30:38.331304 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:30:38.331990 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 8647
I20260809 19:30:38.342723 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 8858
I20260809 19:30:38.351809 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 9065
I20260809 19:30:38.361253 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 8489
2026-08-09T19:30:38Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1964 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[9299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[9299](info): set up 2 sockets
Aug 09 19:30:38 dist-test-slave-x4qh krb5kdc[9299](info): commencing operation
krb5kdc: starting...
W20260809 19:30:39.632306 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 19:30:39.822810 31975 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.190s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 19:30:40.128928 31975 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.306s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 19:30:40.391860 31975 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.263s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/test-user.keytab.
W20260809 19:30:41.004609 31975 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.613s user 0.001s sys 0.001s
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab.
Entry for principal oryx/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab.
I20260809 19:30:41.048193 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33587
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab
--principal=oryx/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:41.122012 9319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:41.122165 9319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:41.122184 9319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:41.123174 9319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:41.123207 9319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:41.123219 9319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:41.123231 9319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:41.123242 9319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:41.124611 9319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab
--principal=oryx/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:41.124782 9319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:41.124930 9319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.129165 9332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.129244 9327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.129263 9319 server_base.cc:1034] running on GCE node
W20260809 19:30:41.129220 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.129735 9319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.129952 9319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.131079 9319 hybrid_clock.cc:648] HybridClock initialized: now 1786303841131060 us; error 36 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.133360 9319 init.cc:377] Logged in from keytab as oryx/127.31.57.254@KRBTEST.COM (short username oryx)
I20260809 19:30:41.133790 9319 webserver.cc:460] Webserver started at http://127.31.57.254:36197/ using document root <none> and password file <none>
I20260809 19:30:41.133998 9319 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.134038 9319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.134187 9319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:41.135108 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "112dfe4504574f149c2ab83d17e096b0"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "bcab80192af4eaf0fec8498a30bf764e"
server_key_iv: "ede2cf17f362cb592dcf30cfdca31973"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.135483 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "112dfe4504574f149c2ab83d17e096b0"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "bcab80192af4eaf0fec8498a30bf764e"
server_key_iv: "ede2cf17f362cb592dcf30cfdca31973"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.136705 9319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:41.137351 9342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.137514 9319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:41.137595 9319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "112dfe4504574f149c2ab83d17e096b0"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "bcab80192af4eaf0fec8498a30bf764e"
server_key_iv: "ede2cf17f362cb592dcf30cfdca31973"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.137671 9319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.144109 9319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.145705 9319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:41.145860 9319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.149489 9319 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:33587
I20260809 19:30:41.149518 9444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:33587 every 8 connection(s)
I20260809 19:30:41.149830 9319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:41.150691 9445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:41.152009 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 9319
I20260809 19:30:41.152120 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:41.152283 31975 external_mini_cluster.cc:1428] Setting key 9681aa3300dec0dad4e263a01a955c64
I20260809 19:30:41.153302 9445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Bootstrap starting.
I20260809 19:30:41.154037 9445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:41.154367 9445 log.cc:824] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:41.155069 9445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: No bootstrap required, opened a new log
I20260809 19:30:41.157135 9445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } }
I20260809 19:30:41.157328 9445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:41.157367 9445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 112dfe4504574f149c2ab83d17e096b0, State: Initialized, Role: FOLLOWER
I20260809 19:30:41.157490 9445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } }
I20260809 19:30:41.157608 9445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:41.157670 9445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:41.157717 9445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:41.158203 9445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } }
I20260809 19:30:41.158350 9445 leader_election.cc:302] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 112dfe4504574f149c2ab83d17e096b0; no voters:
I20260809 19:30:41.158509 9445 leader_election.cc:288] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:41.158542 9456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:41.158707 9456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 LEADER]: Becoming Leader. State: Replica: 112dfe4504574f149c2ab83d17e096b0, State: Running, Role: LEADER
I20260809 19:30:41.158761 9445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:41.158880 9456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } }
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:41.159168 9456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 112dfe4504574f149c2ab83d17e096b0. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } }
I20260809 19:30:41.159178 9462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } }
I20260809 19:30:41.159307 9456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:41.159435 9462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:41.159965 9471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:41.160732 9471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:41.162166 9471 catalog_manager.cc:1347] Generated new cluster ID: 6d5f082248a24ebd91958733f0d2d350
I20260809 19:30:41.162235 9471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:41.163461 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.152820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.31.57.254:33587)
0809 19:30:41.153007 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.153016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.153047 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:30:41.153524 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.153533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.153593 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.153760 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.154336 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.155603 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.156260 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.156908 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.157011 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.159610 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.159614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.159614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.159624 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.161572 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.161955 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.161958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.161958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.162054 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.162306 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.162307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.162307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.162573 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.162674 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.163094 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.163207 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260809 19:30:41.183910 9471 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:41.184422 9471 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:41.188654 9471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Generated new TSK 0
I20260809 19:30:41.188829 9471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
I20260809 19:30:41.219581 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:41.312911 9489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:41.313088 9489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:41.313108 9489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:41.314189 9489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:41.314229 9489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:41.314253 9489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:41.315793 9489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:41.316036 9489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:41.316260 9489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.320984 9500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.320966 9501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.321024 9489 server_base.cc:1034] running on GCE node
W20260809 19:30:41.320962 9504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.321405 9489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.321632 9489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.322795 9489 hybrid_clock.cc:648] HybridClock initialized: now 1786303841322787 us; error 36 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.325306 9489 init.cc:377] Logged in from keytab as oryx/127.31.57.193@KRBTEST.COM (short username oryx)
I20260809 19:30:41.325683 9489 webserver.cc:460] Webserver started at http://127.31.57.193:36219/ using document root <none> and password file <none>
I20260809 19:30:41.325899 9489 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.325964 9489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.326119 9489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:41.327080 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "381a04c485bf641b9ef1f4e6f59c68f3"
server_key_iv: "07c58caf7a1597737aecf6e26f250905"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.327509 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "381a04c485bf641b9ef1f4e6f59c68f3"
server_key_iv: "07c58caf7a1597737aecf6e26f250905"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.329806 9489 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:41.330511 9520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.330682 9489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:41.330750 9489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "381a04c485bf641b9ef1f4e6f59c68f3"
server_key_iv: "07c58caf7a1597737aecf6e26f250905"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.330832 9489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.345407 9489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.347523 9489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:41.347676 9489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.347884 9489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:41.348169 9489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:41.348219 9489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.348250 9489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:41.348275 9489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.353585 9489 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:42867
I20260809 19:30:41.353650 9708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:42867 every 8 connection(s)
I20260809 19:30:41.353991 9489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:41.354699 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 9489
I20260809 19:30:41.354804 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:41.355041 31975 external_mini_cluster.cc:1428] Setting key 12302eeeaf954e31b4dbdeccdfb642d9
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:41.364481 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.354740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:60613 (local address 127.31.57.254:33587)
0809 19:30:41.354802 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.354805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.355054 (+ 249us) server_negotiation.cc:407] Connection header received
0809 19:30:41.355718 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.355719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.355730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.355796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.357059 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.357612 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.358628 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.358861 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.358968 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.361566 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.361567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.361567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.361573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.362898 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.363485 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.363486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.363487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.363537 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.363834 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.363835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.363835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.363940 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.363988 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.364351 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.364419 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:30:41.364780 9711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.354707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:60613)
0809 19:30:41.354916 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.354976 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.355600 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.355868 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.355872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.356685 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.356966 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.356972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.357707 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.357708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.358484 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.358486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.358977 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.358978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.359107 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.359727 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.359735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.361469 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.362985 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.362993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.362995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.363371 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.363605 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.363606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.363606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.363758 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.364005 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.364013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.364182 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.364318 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.364522 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.31.57.194@KRBTEST.COM; defaulting to no policy
I20260809 19:30:41.365355 9709 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:41.365442 9709 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:41.365626 9709 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:41.366180 9378 ts_manager.cc:194] Registered new tserver with Master: e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867)
I20260809 19:30:41.366690 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:60613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
I20260809 19:30:41.380474 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:41.454967 9715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:41.455173 9715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:41.455205 9715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:41.456264 9715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:41.456329 9715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:41.456374 9715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:41.457680 9715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:41.457927 9715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:41.458200 9715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.463016 9723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.463025 9727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.463050 9715 server_base.cc:1034] running on GCE node
W20260809 19:30:41.463037 9724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.463407 9715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.463608 9715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.464744 9715 hybrid_clock.cc:648] HybridClock initialized: now 1786303841464723 us; error 36 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.467023 9715 init.cc:377] Logged in from keytab as oryx/127.31.57.194@KRBTEST.COM (short username oryx)
I20260809 19:30:41.467466 9715 webserver.cc:460] Webserver started at http://127.31.57.194:37621/ using document root <none> and password file <none>
I20260809 19:30:41.467667 9715 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.467724 9715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.467859 9715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:41.468752 9715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "f475f17211444ea680c4a80e8e53803a"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "429424d261191e67e552acb57c6cb790"
server_key_iv: "a97474e6d72801b9f33066116918ceda"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.469111 9715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "f475f17211444ea680c4a80e8e53803a"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "429424d261191e67e552acb57c6cb790"
server_key_iv: "a97474e6d72801b9f33066116918ceda"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.470409 9715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:41.471047 9736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.471231 9715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:41.471302 9715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "f475f17211444ea680c4a80e8e53803a"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "429424d261191e67e552acb57c6cb790"
server_key_iv: "a97474e6d72801b9f33066116918ceda"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.471403 9715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.479943 9715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.481527 9715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:41.481671 9715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.481866 9715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:41.482141 9715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:41.482192 9715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.482228 9715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:41.482254 9715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.488266 9715 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:37039
I20260809 19:30:41.488349 9928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:37039 every 8 connection(s)
I20260809 19:30:41.488656 9715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:41.494596 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 9715
I20260809 19:30:41.494678 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:41.494838 31975 external_mini_cluster.cc:1428] Setting key 68be0ef84b33344dcf78869f56469dba
I20260809 19:30:41.497793 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.489330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42105 (local address 127.31.57.254:33587)
0809 19:30:41.489397 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.489399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.489565 (+ 166us) server_negotiation.cc:407] Connection header received
0809 19:30:41.490028 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.490030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.490043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.490096 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.490922 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.491474 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.492440 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.492645 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.492723 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.494802 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.494803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.494803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.494811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.496361 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.496761 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.496762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.496763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.496830 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.497145 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.497147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.497147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.497275 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.497346 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.497671 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.497736 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:30:41.498051 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.489302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:42105)
0809 19:30:41.489455 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.489486 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.489922 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.490162 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.490165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.490653 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.490819 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.490823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.491579 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.491580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.492331 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.492333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.492730 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.492731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.492853 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.493331 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.493339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.494687 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.496463 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.496471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.496473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.496682 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.496911 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.496912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.496912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.497054 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.497358 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.497364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.497509 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.497619 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.497775 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260809 19:30:41.498477 9930 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:41.498569 9930 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:41.498735 9930 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:41.499065 9378 ts_manager.cc:194] Registered new tserver with Master: f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
I20260809 19:30:41.499471 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:42105
WARNING: no policy specified for oryx/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
I20260809 19:30:41.521435 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:41.599521 9942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:41.599699 9942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:41.599720 9942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:41.600733 9942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:41.600766 9942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:41.600790 9942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:41.602015 9942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:41.602173 9942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:41.602335 9942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.607211 9951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.607164 9960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.607158 9956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.607436 9942 server_base.cc:1034] running on GCE node
I20260809 19:30:41.607735 9942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.607985 9942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.609138 9942 hybrid_clock.cc:648] HybridClock initialized: now 1786303841609124 us; error 28 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.611578 9942 init.cc:377] Logged in from keytab as oryx/127.31.57.195@KRBTEST.COM (short username oryx)
I20260809 19:30:41.612051 9942 webserver.cc:460] Webserver started at http://127.31.57.195:43261/ using document root <none> and password file <none>
I20260809 19:30:41.612277 9942 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.612342 9942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.612485 9942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:41.613454 9942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "fa79d60dbe39406f81dba806f3431da5"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c980861121e191a7bdfb12000c95753f"
server_key_iv: "64b90e517a96587bb754e295e58e91d9"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.613834 9942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "fa79d60dbe39406f81dba806f3431da5"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c980861121e191a7bdfb12000c95753f"
server_key_iv: "64b90e517a96587bb754e295e58e91d9"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.615168 9942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:30:41.615939 9970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.616102 9942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.616199 9942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "fa79d60dbe39406f81dba806f3431da5"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c980861121e191a7bdfb12000c95753f"
server_key_iv: "64b90e517a96587bb754e295e58e91d9"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.616278 9942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.630048 9942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.632181 9942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:41.632344 9942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.632568 9942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:41.632889 9942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:41.632943 9942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.632982 9942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:41.633009 9942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.638800 9942 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:37301
I20260809 19:30:41.638867 10155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:37301 every 8 connection(s)
I20260809 19:30:41.639174 9942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:41.646243 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 9942
I20260809 19:30:41.646317 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:41.646472 31975 external_mini_cluster.cc:1428] Setting key e3aaac3b0bcbbb8d97d1382a26bf5f15
I20260809 19:30:41.647934 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.639882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:58945 (local address 127.31.57.254:33587)
0809 19:30:41.639937 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.639939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.640127 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:30:41.640581 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.640583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.640595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.640652 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.641548 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.642018 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.642930 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.643149 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.643223 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.645214 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.645216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.645216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.645223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.646409 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.646908 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.646910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.646911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.646974 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.647246 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.647247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.647247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.647429 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.647491 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.647762 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.647844 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:30:41.648109 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.639876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:58945)
0809 19:30:41.640024 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.640054 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.640483 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.640712 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.640715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.641286 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.641450 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.641454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.642111 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.642112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.642836 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.642839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.643242 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.643243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.643392 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.643884 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.643892 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.645123 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.646506 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.646513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.646515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.646832 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.647052 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.647053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.647054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.647171 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.647483 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.647489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.647632 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.647737 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.647892 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 19:30:41.648568 10156 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:41.648667 10156 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:41.648829 10156 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:41.649165 9378 ts_manager.cc:194] Registered new tserver with Master: fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:41.649528 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:58945
I20260809 19:30:41.658129 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.57.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.253.keytab.
Entry for principal oryx/127.31.57.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.253.keytab.
I20260809 19:30:41.682228 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:43035
--webserver_interface=127.31.57.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.253.keytab
--principal=oryx/127.31.57.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33587,127.31.57.253:43035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:41.758248 10163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:41.758422 10163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:41.758442 10163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:41.759588 10163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:41.759665 10163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:41.759697 10163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:41.759725 10163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:41.759742 10163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:41.761243 10163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33587,127.31.57.253:43035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.253.keytab
--principal=oryx/127.31.57.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:43035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.57.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
W20260809 19:30:41.761364 10163 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.57.254:33587,127.31.57.253:43035'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 19:30:41.761499 10163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:41.761718 10163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.766305 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.766316 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.766323 10172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.766507 10163 server_base.cc:1034] running on GCE node
I20260809 19:30:41.766722 10163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.766974 10163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.768115 10163 hybrid_clock.cc:648] HybridClock initialized: now 1786303841768095 us; error 38 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.770555 10163 init.cc:377] Logged in from keytab as oryx/127.31.57.253@KRBTEST.COM (short username oryx)
I20260809 19:30:41.770918 10163 webserver.cc:460] Webserver started at http://127.31.57.253:44035/ using document root <none> and password file <none>
I20260809 19:30:41.771085 10163 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.771135 10163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.771250 10163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:41.772205 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/instance:
uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c6f36167af7e3924c8369832beaa4f9f"
server_key_iv: "7a9c00cb003ef76dd0d45e4a7cba96da"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.772486 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal/instance:
uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c6f36167af7e3924c8369832beaa4f9f"
server_key_iv: "7a9c00cb003ef76dd0d45e4a7cba96da"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.773762 10163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:41.774439 10191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.774626 10163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:41.774698 10163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c6f36167af7e3924c8369832beaa4f9f"
server_key_iv: "7a9c00cb003ef76dd0d45e4a7cba96da"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.774745 10163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.784772 10163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.786893 10163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:41.787024 10163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.791014 10163 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.253:43035
I20260809 19:30:41.791082 10282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.253:43035 every 8 connection(s)
I20260809 19:30:41.791460 10163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
I20260809 19:30:41.792395 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:41.793687 10285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:41.796370 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 10163
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.253@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
I20260809 19:30:41.802491 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.794349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.31.57.254:33587)
0809 19:30:41.794413 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.794415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.794568 (+ 153us) server_negotiation.cc:407] Connection header received
0809 19:30:41.794979 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.794980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.794989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.795046 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.795668 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.796144 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.797247 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.797443 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.797512 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.799632 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.799633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.799634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.799641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.801110 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.801511 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.801512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.801512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.801596 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.801841 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.801842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.801842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.801990 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.802063 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.802323 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.802406 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:30:41.802624 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.794285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35032)
0809 19:30:41.794450 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.794487 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.794899 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.795094 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.795097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.795328 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.795586 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.795590 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.796242 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.796243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.797156 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.797159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.797528 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.797529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.797633 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.798107 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.798115 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.799553 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.801212 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.801218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.801220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.801431 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.801657 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.801658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.801658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.801770 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.802065 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.802070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.802190 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.802282 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.802409 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260809 19:30:41.803337 10285 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:41.805294 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.796852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.31.57.253:43035)
0809 19:30:41.797010 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.797014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.797037 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:30:41.797094 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.797096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.797115 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.797183 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.797751 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.798431 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.799461 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.799852 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.799916 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.801966 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.801973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.801973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.801979 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.804084 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.804446 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.804449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.804449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.804509 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.804801 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.804802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.804802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.804955 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.805101 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.805176 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.805230 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.253@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
I20260809 19:30:41.810187 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.803615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37704)
0809 19:30:41.803655 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.803665 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.803703 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.803916 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.803917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.804096 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.804165 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.804166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.804972 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.804973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.805916 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.805919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.806209 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.806210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.806254 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.806520 (+ 266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.806523 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.807637 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.809407 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.809408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.809408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.809586 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.809751 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.809752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.809752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.809806 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.809995 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.809996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.810026 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.810070 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.810136 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:30:41.810302 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.803645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.31.57.253:43035)
0809 19:30:41.803776 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.803779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.803796 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:30:41.803816 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.803817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.803827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.803905 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.804243 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.804898 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.805980 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.806151 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.806200 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.807718 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.807719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.807720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.807725 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.809339 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.809648 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.809649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.809649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.809697 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.809857 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.809858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.809858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.809944 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.809991 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.810091 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.810178 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 19:30:41.811443 10285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Bootstrap starting.
I20260809 19:30:41.812014 10285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:41.812264 10285 log.cc:824] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:41.812968 10285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: No bootstrap required, opened a new log
I20260809 19:30:41.814205 10285 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } }
I20260809 19:30:41.814333 10285 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:41.814379 10285 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb2712f97384bb3a9265c4bb59ee1c4, State: Initialized, Role: FOLLOWER
I20260809 19:30:41.814520 10285 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } }
I20260809 19:30:41.814815 10285 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:41.814795 10293 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } }
I20260809 19:30:41.814908 10293 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:30:41.816540 10316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:30:41.816609 10316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:30:41.817976 10163 master_runner.cc:428] Detected that this master feb2712f97384bb3a9265c4bb59ee1c4 is joining an existing cluster
I20260809 19:30:41.818029 10163 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.57.253:43035
I20260809 19:30:41.824901 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.818995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.31.57.254:33587)
0809 19:30:41.819029 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.819031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.819198 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:30:41.819343 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.819344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.819354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.819457 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.819837 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.820258 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.820930 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.821103 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.821179 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.822047 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.822048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.822048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.822055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.823683 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.824096 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.824098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.824098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.824152 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.824377 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.824377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.824378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.824487 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.824561 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.824739 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.824811 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:30:41.824918 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.818969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35038)
0809 19:30:41.819122 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.819130 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.819224 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.819464 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.819465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.819687 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.819759 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.819760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.820344 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.820345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.820840 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.820842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.821204 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.821205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.821260 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.821642 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.821646 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.821939 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.823766 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.823767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.823767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.824002 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.824227 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.824228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.824228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.824286 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.824553 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.824554 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.824606 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.824710 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.824818 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260809 19:30:41.826015 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.819186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37714)
0809 19:30:41.819358 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.819367 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.819450 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.819580 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.819580 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.819817 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.819884 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.819886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.820662 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.820663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.821424 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.821426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.821915 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.821916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.821969 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.822384 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.822388 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.822707 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.824994 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.824995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.824995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.825159 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.825371 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.825371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.825372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.825425 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.825660 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.825661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.825696 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.825817 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.825907 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260809 19:30:41.826033 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.819221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.31.57.253:43035)
0809 19:30:41.819251 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.819253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.819429 (+ 176us) server_negotiation.cc:407] Connection header received
0809 19:30:41.819518 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.819519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.819528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.819576 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.819964 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.820581 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.821508 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.821813 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.821852 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.822816 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.822817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.822818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.822824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.824476 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.825246 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.825247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.825247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.825308 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.825496 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.825497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.825497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.825597 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.825668 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.825839 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.825954 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":16}
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.254@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
I20260809 19:30:41.837272 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.827506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.31.57.253:43035)
0809 19:30:41.827536 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.827537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.830652 (+ 3115us) server_negotiation.cc:407] Connection header received
0809 19:30:41.830734 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.830735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.830744 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.830804 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.831176 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.831640 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.832411 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.832594 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.832668 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.834841 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.834842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.834842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.834848 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.836181 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.836590 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.836590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.836591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.836635 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.836837 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.836838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.836838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.836921 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.836967 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.837167 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.837222 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:30:41.837352 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.827474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37720)
0809 19:30:41.830560 (+ 3086us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.830589 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.830674 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.830809 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.830813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.831047 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.831089 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.831090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.831714 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.831716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.832306 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.832309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.832682 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.832683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.832762 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.833223 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.833235 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.834730 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.836258 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.836265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.836267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.836520 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.836699 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.836700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.836700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.836767 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.836982 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.836989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.837040 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.837143 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.837255 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3064,"thread_start_us":57,"threads_started":1}
I20260809 19:30:41.837976 9378 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.57.253:43035: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
member_type: NON_VOTER
last_known_addr {
host: "127.31.57.253"
port: 43035
}
attrs {
promote: true
}
}
dest_uuid: "112dfe4504574f149c2ab83d17e096b0"
cas_config_opid_index: -1
I20260809 19:30:41.838328 9378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } }
I20260809 19:30:41.838801 10330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:41.839123 10242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:41.839926 10330 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.57.253:43035
I20260809 19:30:41.840093 10242 raft_consensus.cc:1261] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 19:30:41.840193 10328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:41.840246 10163 master.cc:537] Master@127.31.57.253:43035 shutting down...
I20260809 19:30:41.840291 10328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:41.840512 10328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:30:41.841233 9348 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 19:30:41.841996 9348 consensus_peers.cc:487] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 -> Peer feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253:43035): Couldn't send request to peer feb2712f97384bb3a9265c4bb59ee1c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:30:41.842990 10163 raft_consensus.cc:2229] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:41.843068 10163 raft_consensus.cc:2258] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:41.843102 10163 tablet_replica.cc:330] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: stopping tablet replica
I20260809 19:30:41.855821 10163 master.cc:552] Master@127.31.57.253:43035 shutdown complete.
I20260809 19:30:41.855867 10163 master_runner.cc:305] Clearing existing system tablet
I20260809 19:30:41.855988 10163 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.856033 10163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.856657 10163 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:41.857132 10335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.857252 10163 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.857312 10163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.857348 10163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c6f36167af7e3924c8369832beaa4f9f"
server_key_iv: "7a9c00cb003ef76dd0d45e4a7cba96da"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.857928 10163 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:41.859166 10163 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:30:41.859222 10163 log.cc:1196] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 19:30:41.859475 10163 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Deleting consensus metadata
I20260809 19:30:41.859603 10163 master_runner.cc:315] Copying system tablet from 127.31.57.254:33587
I20260809 19:30:41.860230 10163 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: tablet copy: Beginning tablet copy session from remote peer at address 127.31.57.254:33587
I20260809 19:30:41.866110 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.860589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.31.57.254:33587)
0809 19:30:41.860646 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.860648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.860769 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:30:41.860850 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.860851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.860860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.860911 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.861187 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.861613 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.862324 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.862505 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.862570 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.863512 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.863513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.863513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.863520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.864954 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.865322 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.865323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.865324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.865381 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.865602 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.865603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.865604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.865713 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.865801 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.865953 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.866025 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:41.866146 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.860549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35046)
0809 19:30:41.860695 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:41.860704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:41.860791 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:41.860916 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:41.860916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:41.861069 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:41.861115 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.861116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.861688 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.861689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.862229 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:41.862231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.862569 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.862570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:41.862631 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.863058 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:41.863062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:41.863336 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:41.865059 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.865060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.865061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.865243 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.865470 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:41.865471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:41.865471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:41.865527 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:41.865797 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:41.865798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:41.865840 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:30:41.865928 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:30:41.866026 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":58,"threads_started":1}
I20260809 19:30:41.866760 9420 tablet_copy_service.cc:140] P 112dfe4504574f149c2ab83d17e096b0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer feb2712f97384bb3a9265c4bb59ee1c4 ({username='oryx', principal='oryx/127.31.57.253@KRBTEST.COM'} at 127.0.0.1:35046)
I20260809 19:30:41.866866 9420 tablet_copy_service.cc:161] P 112dfe4504574f149c2ab83d17e096b0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer feb2712f97384bb3a9265c4bb59ee1c4 at {username='oryx', principal='oryx/127.31.57.253@KRBTEST.COM'} at 127.0.0.1:35046: session id = feb2712f97384bb3a9265c4bb59ee1c4-00000000000000000000000000000000
I20260809 19:30:41.867726 9420 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:30:41.868355 10163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:30:41.869225 10163 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: tablet copy: Starting download of 0 data blocks...
I20260809 19:30:41.869405 10163 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: tablet copy: Starting download of 1 WAL segments...
I20260809 19:30:41.870581 10163 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:30:41.871726 9420 tablet_copy_service.cc:342] P 112dfe4504574f149c2ab83d17e096b0: Request end of tablet copy session feb2712f97384bb3a9265c4bb59ee1c4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.57.253@KRBTEST.COM'} at 127.0.0.1:35046
I20260809 19:30:41.871790 9420 tablet_copy_service.cc:433] P 112dfe4504574f149c2ab83d17e096b0: ending tablet copy session feb2712f97384bb3a9265c4bb59ee1c4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer feb2712f97384bb3a9265c4bb59ee1c4
I20260809 19:30:41.874298 10163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:41.875336 10345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.875351 10346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:41.875504 10348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:41.875793 10163 server_base.cc:1034] running on GCE node
I20260809 19:30:41.875887 10163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:41.876037 10163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:41.877202 10163 hybrid_clock.cc:648] HybridClock initialized: now 1786303841877199 us; error 31 us; skew 500 ppm
Aug 09 19:30:41 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:41.878549 10163 init.cc:377] Logged in from keytab as oryx/127.31.57.253@KRBTEST.COM (short username oryx)
I20260809 19:30:41.878886 10163 webserver.cc:460] Webserver started at http://127.31.57.253:37137/ using document root <none> and password file <none>
I20260809 19:30:41.879035 10163 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:41.879074 10163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:41.879729 10163 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:41.880254 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:41.880383 10163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:41.880427 10163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c6f36167af7e3924c8369832beaa4f9f"
server_key_iv: "7a9c00cb003ef76dd0d45e4a7cba96da"
server_key_version: "encryptionkey@0"
I20260809 19:30:41.880474 10163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:41.899724 10163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:41.901340 10163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:41.904247 10163 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.253:43035
I20260809 19:30:41.904265 10449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.253:43035 every 8 connection(s)
I20260809 19:30:41.904572 10163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
I20260809 19:30:41.905387 10450 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:30:41.905895 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Bootstrap starting.
I20260809 19:30:41.908638 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:30:41.908854 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: Bootstrap complete.
I20260809 19:30:41.909042 10450 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } }
I20260809 19:30:41.909135 10450 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: feb2712f97384bb3a9265c4bb59ee1c4, State: Initialized, Role: LEARNER
I20260809 19:30:41.909168 10450 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } }
I20260809 19:30:41.909348 10450 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:41.909341 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:41.909574 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:30:41.910706 10472 catalog_manager.cc:1259] Loaded cluster ID: 6d5f082248a24ebd91958733f0d2d350
I20260809 19:30:41.910786 10472 catalog_manager.cc:1552] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: loading cluster ID for follower catalog manager: success
I20260809 19:30:41.911999 10472 catalog_manager.cc:1574] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: acquiring CA information for follower catalog manager: success
I20260809 19:30:41.912317 10472 catalog_manager.cc:1602] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:30:41.913431 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:41.906622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37738 (local address 127.31.57.253:43035)
0809 19:30:41.906794 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:41.906795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:41.906801 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:30:41.906819 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:41.906819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:41.906827 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:41.906893 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:41.907550 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.908158 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.908835 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:41.909086 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:41.909159 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:41.910518 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:41.910519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:41.910519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:41.910525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:41.912235 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.912605 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.912606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.912606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.912675 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:41.912860 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:41.912861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:41.912861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:41.912973 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:41.913096 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:41.913211 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 19:30:41.913296 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.31.57.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.252.keytab.
Entry for principal oryx/127.31.57.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.252.keytab.
I20260809 19:30:41.937948 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:40479
--webserver_interface=127.31.57.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.252.keytab
--principal=oryx/127.31.57.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:42.017577 10476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:42.017796 10476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:42.017833 10476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:42.019044 10476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:42.019100 10476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:42.019126 10476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:42.019156 10476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:42.019179 10476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:42.020749 10476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.252.keytab
--principal=oryx/127.31.57.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:40479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.57.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:42.021015 10476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:42.021226 10476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:42.025874 10483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.026048 10487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.025916 10476 server_base.cc:1034] running on GCE node
W20260809 19:30:42.025916 10484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.026407 10476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:42.026680 10476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:42.027829 10476 hybrid_clock.cc:648] HybridClock initialized: now 1786303842027818 us; error 42 us; skew 500 ppm
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:42.030228 10476 init.cc:377] Logged in from keytab as oryx/127.31.57.252@KRBTEST.COM (short username oryx)
I20260809 19:30:42.030625 10476 webserver.cc:460] Webserver started at http://127.31.57.252:45037/ using document root <none> and password file <none>
I20260809 19:30:42.030836 10476 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.030901 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.031041 10476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:42.032109 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/instance:
uuid: "1fa75af971344d12ad205379acc506b7"
format_stamp: "Formatted at 2026-08-09 19:30:42 on dist-test-slave-x4qh"
server_key: "e5a2ce16886e7e34be427b85a966cd04"
server_key_iv: "eb64a6c9904246567c2932b6c98100e4"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.032558 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal/instance:
uuid: "1fa75af971344d12ad205379acc506b7"
format_stamp: "Formatted at 2026-08-09 19:30:42 on dist-test-slave-x4qh"
server_key: "e5a2ce16886e7e34be427b85a966cd04"
server_key_iv: "eb64a6c9904246567c2932b6c98100e4"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.033886 10476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:42.034581 10495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.034731 10476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:42.034824 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "1fa75af971344d12ad205379acc506b7"
format_stamp: "Formatted at 2026-08-09 19:30:42 on dist-test-slave-x4qh"
server_key: "e5a2ce16886e7e34be427b85a966cd04"
server_key_iv: "eb64a6c9904246567c2932b6c98100e4"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.034897 10476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.055099 10476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:42.056734 10476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:42.056890 10476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:42.061187 10476 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.252:40479
I20260809 19:30:42.061218 10573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.252:40479 every 8 connection(s)
I20260809 19:30:42.061518 10476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
I20260809 19:30:42.062433 10575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:42.062826 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 10476
I20260809 19:30:42.064673 10575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.252@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:42.073244 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.063431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.31.57.252:40479)
0809 19:30:42.063573 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.063580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.063601 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:30:42.064027 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.064033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.064075 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.064211 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.064836 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.066044 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.067038 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.067595 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.067691 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.070069 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.070075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.070075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.070080 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.071840 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072166 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.072169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.072169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.072258 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072448 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.072448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.072449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.072722 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.072840 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.072929 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.073060 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260809 19:30:42.073307 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.065447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.31.57.254:33587)
0809 19:30:42.065488 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.065489 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.065559 (+ 70us) server_negotiation.cc:407] Connection header received
0809 19:30:42.065621 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.065622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.065631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.065677 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.066188 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.066899 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.067894 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.068071 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.068134 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.070684 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.070686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.070686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.070694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.072076 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072524 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.072525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.072525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.072579 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072743 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.072744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.072744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.072881 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.072944 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.073133 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.073209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:42.073318 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.065287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35054)
0809 19:30:42.065451 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.065488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.065561 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.065700 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.065702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.066022 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.066103 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.066105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.066991 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.066992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.067810 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.067812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.068161 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.068163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.068224 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.068656 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.068663 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.070598 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.072206 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.072213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.072457 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.072638 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.072639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.072639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.072686 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.072948 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.072955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.073005 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.073090 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.073196 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260809 19:30:42.073758 10575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.252@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
I20260809 19:30:42.078876 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.074040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37746)
0809 19:30:42.074087 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.074098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.074137 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.074319 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.074320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.074421 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.074464 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.074465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.075039 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.075040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.075534 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.075535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.075832 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.075833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.075878 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.076147 (+ 269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.076150 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.077035 (+ 885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.078192 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.078193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.078193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.078347 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.078501 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.078502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.078502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.078548 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.078694 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.078695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.078721 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.078762 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.078826 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:30:42.078897 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.074154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.31.57.253:43035)
0809 19:30:42.074209 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.074211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.074220 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:42.074243 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.074244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.074259 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.074310 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.074505 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.074968 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.075603 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.075779 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.075824 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.077103 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.077104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.077104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.077109 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.078125 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.078405 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.078406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.078406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.078450 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.078582 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.078582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.078583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.078659 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.078699 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.078775 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.078843 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:30:42.079294 10575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.252@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
I20260809 19:30:42.083957 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.079530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35884)
0809 19:30:42.079574 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.079584 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.079620 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.079715 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.079716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.079816 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.079839 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.079840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.080378 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.080379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.080775 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.080776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.081059 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.081059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.081091 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.081383 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.081388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.082296 (+ 908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.083298 (+ 1002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.083299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.083299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.083475 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.083600 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.083600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.083600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.083637 (+ 37us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.083761 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.083762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.083795 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.083851 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.083910 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:30:42.084065 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.079578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.31.57.252:40479)
0809 19:30:42.079614 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.079615 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.079622 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:42.079668 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.079669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.079675 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.079707 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.079877 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.080251 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.080843 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.081007 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.081051 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.082358 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.082359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.082359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.082364 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.083240 (+ 876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.083527 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.083527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.083527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.083566 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.083666 (+ 100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.083666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.083666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.083733 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.083771 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.083866 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.083950 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":18}
I20260809 19:30:42.085249 10575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Bootstrap starting.
I20260809 19:30:42.085922 10575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:42.086177 10575 log.cc:824] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:42.086769 10575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: No bootstrap required, opened a new log
I20260809 19:30:42.088006 10575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } }
I20260809 19:30:42.088105 10575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:42.088125 10575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa75af971344d12ad205379acc506b7, State: Initialized, Role: FOLLOWER
I20260809 19:30:42.088207 10575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } }
I20260809 19:30:42.088430 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } } }
I20260809 19:30:42.088490 10575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:42.088513 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:30:42.090018 10597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:30:42.090073 10597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:30:42.092018 10476 master_runner.cc:428] Detected that this master 1fa75af971344d12ad205379acc506b7 is joining an existing cluster
I20260809 19:30:42.092067 10476 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.57.252:40479
I20260809 19:30:42.098912 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.092934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.31.57.254:33587)
0809 19:30:42.092975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.092977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.093111 (+ 134us) server_negotiation.cc:407] Connection header received
0809 19:30:42.093199 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.093200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.093210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.093267 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.093648 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094351 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095417 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.095677 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095720 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.096770 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.096771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.096771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.096777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.097983 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.098326 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.098326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.098326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.098368 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.098529 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.098529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.098529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.098607 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.098648 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.098815 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.098863 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:42.098973 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.092924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35060)
0809 19:30:42.093056 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.093065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.093138 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.093312 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.093313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.093497 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.093558 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.093559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094449 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.094450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.095298 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.095300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.095811 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.095872 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.096288 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.096292 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.096679 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.098058 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.098059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.098059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.098259 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.098408 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.098409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.098410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.098474 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.098677 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.098677 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.098715 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.098804 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.098887 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303841, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.254@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
I20260809 19:30:42.122471 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.100674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35888)
0809 19:30:42.100718 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.100725 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.100757 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.100962 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.100963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.101088 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.101116 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.101117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.101704 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.101705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.102156 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.102158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.102477 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.102478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.102537 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.102869 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.102873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.103917 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.121324 (+ 17407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.121325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.121326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.121532 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.121800 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.121800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.121801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.121865 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.122160 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.122161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.122204 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.122249 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.122357 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:30:42.123109 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.100714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.31.57.252:40479)
0809 19:30:42.100856 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.100857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.100862 (+ 5us) server_negotiation.cc:407] Connection header received
0809 19:30:42.100885 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.100885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.100892 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.100948 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.101202 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.101631 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.102226 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.102414 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.102476 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.104017 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.104018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.104019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.104024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.121225 (+ 17201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.121613 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.121614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.121614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.121667 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.121940 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.121941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.121941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.122078 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.122875 (+ 797us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.122913 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.122988 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 19:30:42.123600 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.093332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.31.57.252:40479)
0809 19:30:42.093361 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.093363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.093447 (+ 84us) server_negotiation.cc:407] Connection header received
0809 19:30:42.093524 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.093525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.093533 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.093558 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.093957 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094612 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095632 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.095868 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095911 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.120795 (+ 24884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.120796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.120796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.120802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.122322 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.122644 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.122645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.122645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.122915 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.123080 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.123080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.123081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.123186 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.123229 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.123441 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.123516 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:30:42.123706 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.092970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.31.57.253:43035)
0809 19:30:42.093267 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.093269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.093307 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:30:42.093402 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.093403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.093414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.093445 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.093959 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094628 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095638 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.095917 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095950 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.120627 (+ 24677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.120629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.120629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.120637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.122525 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.122886 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.122887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.122887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.122944 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.123136 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.123137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.123137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.123263 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.123320 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.123545 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.123619 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277}
I20260809 19:30:42.123836 9378 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.57.252:40479: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1fa75af971344d12ad205379acc506b7"
member_type: NON_VOTER
last_known_addr {
host: "127.31.57.252"
port: 40479
}
attrs {
promote: true
}
}
dest_uuid: "112dfe4504574f149c2ab83d17e096b0"
cas_config_opid_index: 5
I20260809 19:30:42.124073 9378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
I20260809 19:30:42.124352 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.093033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37758)
0809 19:30:42.093255 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.093265 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.093330 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.093507 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.093508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.093795 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.093866 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.093868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094709 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.094710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.095549 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.095552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.096006 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.096007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.096054 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.119933 (+ 23879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.119937 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.120525 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.122616 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.122617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.122618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.122815 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.123019 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.123020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.123020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.123075 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.123344 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.123345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.123403 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.123461 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.124242 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":676,"thread_start_us":72,"threads_started":1}
I20260809 19:30:42.124413 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 9489
I20260809 19:30:42.124526 10330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1fa75af971344d12ad205379acc506b7 (127.31.57.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.125677 10330 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.57.252:40479
I20260809 19:30:42.125746 10330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.125846 10330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.126196 10531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:42.127333 10531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:30:42.127626 10328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:42.129900 10531 raft_consensus.cc:858] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
W20260809 19:30:42.129995 10531 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
I20260809 19:30:42.131814 10531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:42.133271 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.128842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37760)
0809 19:30:42.129756 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.129766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.129815 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.130001 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.130002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.130003 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.130043 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.130045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.130889 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.130890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.132023 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.132027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.133031 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.133032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.133099 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.133122 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.133170 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.133206 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901}
I20260809 19:30:42.133416 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 112dfe4504574f149c2ab83d17e096b0. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:42.133519 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 19:30:42.133590 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.128969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.31.57.253:43035)
0809 19:30:42.129729 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.129731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.129859 (+ 128us) server_negotiation.cc:407] Connection header received
0809 19:30:42.129881 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.129881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.129902 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.129951 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.130104 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.130798 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.132297 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.132869 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.132930 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.132998 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.133469 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.133534 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742}
I20260809 19:30:42.133569 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:42.133634 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 19:30:42.133678 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } }
I20260809 19:30:42.133741 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 19:30:42.133905 10405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 LEARNER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:30:42.134276 10615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 19:30:42.134930 10405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1fa75af971344d12ad205379acc506b7 (127.31.57.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.136089 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 112dfe4504574f149c2ab83d17e096b0. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.136170 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:30:42.136202 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: NON_VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: true } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.136242 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:30:42.139900 10615 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: attempting to promote NON_VOTER feb2712f97384bb3a9265c4bb59ee1c4 to VOTER
I20260809 19:30:42.140123 10615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
I20260809 19:30:42.143023 10405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 LEARNER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 19:30:42.143114 10476 master.cc:537] Master@127.31.57.252:40479 shutting down...
I20260809 19:30:42.143973 10328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 19:30:42.144375 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.093154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35886)
0809 19:30:42.093400 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.093408 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.093464 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.093617 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.093617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.093833 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.093886 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.093887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.094709 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.094710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.095551 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.095552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.095970 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.095971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.096023 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.120208 (+ 24185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.120212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.120695 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.122406 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.122406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.122407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.122578 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.122972 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.122973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.122973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.123022 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.123263 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.123263 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.123314 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.123698 (+ 384us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.123810 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":42,"thread_start_us":67,"threads_started":1}
I20260809 19:30:42.145123 10616 mvcc.cc:205] Tried to move back new op lower bound from 7316700533507301376 to 7316700533387071488. Current Snapshot: MvccSnapshot[applied={T|T < 7316700533397479424}]
I20260809 19:30:42.146433 10530 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260809 19:30:42.146522 10530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 19:30:42.146908 10328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 19:30:42.147058 10328 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.147204 10405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.148039 10328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.148123 10328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.148188 10328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.148244 10328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.148237 10476 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 19:30:42.148346 10476 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 19:30:42.148384 10476 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: stopping tablet replica
I20260809 19:30:42.148679 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.148871 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:30:42.148922 9344 consensus_peers.cc:487] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 -> Peer 1fa75af971344d12ad205379acc506b7 (127.31.57.252:40479): Couldn't send request to peer 1fa75af971344d12ad205379acc506b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:30:42.150543 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:42867
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:33587
--webserver_port=36219
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:30:42.153223 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.153091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35894)
0809 19:30:42.153175 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.153202 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:30:42.163681 10476 master.cc:552] Master@127.31.57.252:40479 shutdown complete.
I20260809 19:30:42.163727 10476 master_runner.cc:305] Clearing existing system tablet
I20260809 19:30:42.163889 10476 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.163935 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.164669 10476 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.165220 10633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.165369 10476 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.165423 10476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.165447 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "1fa75af971344d12ad205379acc506b7"
format_stamp: "Formatted at 2026-08-09 19:30:42 on dist-test-slave-x4qh"
server_key: "e5a2ce16886e7e34be427b85a966cd04"
server_key_iv: "eb64a6c9904246567c2932b6c98100e4"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.166149 10476 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:42.167496 10476 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:30:42.167565 10476 log.cc:1196] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 19:30:42.167865 10476 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Deleting consensus metadata
I20260809 19:30:42.167996 10476 master_runner.cc:315] Copying system tablet from 127.31.57.254:33587
I20260809 19:30:42.168601 10476 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: tablet copy: Beginning tablet copy session from remote peer at address 127.31.57.254:33587
I20260809 19:30:42.175693 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.168867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35062)
0809 19:30:42.168975 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.168984 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.169038 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.169181 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.169181 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.169376 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.169427 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.169429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.170300 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.170301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.171045 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.171047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.171632 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.171633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.171688 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.172190 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.172195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.172548 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.174467 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.174468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.174469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.174707 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.174981 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.174982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.174983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.175052 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.175358 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.175359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.175435 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.175506 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.175600 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260809 19:30:42.175706 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.168936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.31.57.254:33587)
0809 19:30:42.168978 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.168980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.169026 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:30:42.169089 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.169090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.169101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.169174 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.169527 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.170212 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.171132 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.171467 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.171539 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.172641 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.172643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.172643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.172651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.174388 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.174835 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.174837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.174837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.174903 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.175141 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.175142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.175143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.175287 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.175360 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.175553 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.175629 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:30:42.176343 9420 tablet_copy_service.cc:140] P 112dfe4504574f149c2ab83d17e096b0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1fa75af971344d12ad205379acc506b7 ({username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:35062)
I20260809 19:30:42.176405 9420 tablet_copy_service.cc:161] P 112dfe4504574f149c2ab83d17e096b0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1fa75af971344d12ad205379acc506b7 at {username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:35062: session id = 1fa75af971344d12ad205379acc506b7-00000000000000000000000000000000
I20260809 19:30:42.176970 9420 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:30:42.177402 10476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:30:42.178515 10476 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: tablet copy: Starting download of 0 data blocks...
I20260809 19:30:42.178696 10476 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: tablet copy: Starting download of 1 WAL segments...
I20260809 19:30:42.179833 10476 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:30:42.180712 9420 tablet_copy_service.cc:342] P 112dfe4504574f149c2ab83d17e096b0: Request end of tablet copy session 1fa75af971344d12ad205379acc506b7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:35062
I20260809 19:30:42.180763 9420 tablet_copy_service.cc:433] P 112dfe4504574f149c2ab83d17e096b0: ending tablet copy session 1fa75af971344d12ad205379acc506b7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1fa75af971344d12ad205379acc506b7
I20260809 19:30:42.183352 10476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:42.184556 10648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.184593 10642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.185159 10476 server_base.cc:1034] running on GCE node
W20260809 19:30:42.187525 10643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.187690 10476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:42.187845 10476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:42.189003 10476 hybrid_clock.cc:648] HybridClock initialized: now 1786303842188993 us; error 22 us; skew 500 ppm
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:42.190510 10476 init.cc:377] Logged in from keytab as oryx/127.31.57.252@KRBTEST.COM (short username oryx)
I20260809 19:30:42.190748 10476 webserver.cc:460] Webserver started at http://127.31.57.252:45573/ using document root <none> and password file <none>
I20260809 19:30:42.190855 10476 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.190887 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.191617 10476 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:30:42.192211 10659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.192385 10476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:42.192436 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "1fa75af971344d12ad205379acc506b7"
format_stamp: "Formatted at 2026-08-09 19:30:42 on dist-test-slave-x4qh"
server_key: "e5a2ce16886e7e34be427b85a966cd04"
server_key_iv: "eb64a6c9904246567c2932b6c98100e4"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.192504 10476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.206748 10476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:42.208438 10476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:42.211444 10476 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.252:40479
I20260809 19:30:42.211555 10740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.252:40479 every 8 connection(s)
I20260809 19:30:42.211772 10476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
I20260809 19:30:42.212685 10741 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:30:42.213279 10741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Bootstrap starting.
I20260809 19:30:42.215519 10741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:30:42.215731 10741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Bootstrap complete.
I20260809 19:30:42.215941 10741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
I20260809 19:30:42.216023 10741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1fa75af971344d12ad205379acc506b7, State: Initialized, Role: LEARNER
I20260809 19:30:42.216056 10741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } }
I20260809 19:30:42.216226 10741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:42.217555 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.217628 10764 catalog_manager.cc:1259] Loaded cluster ID: 6d5f082248a24ebd91958733f0d2d350
I20260809 19:30:42.217653 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:30:42.218449 10764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: loading cluster ID for follower catalog manager: success
I20260809 19:30:42.219646 10764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: acquiring CA information for follower catalog manager: success
I20260809 19:30:42.219983 10764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 19:30:42.244760 10630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:42.244946 10630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:42.244967 10630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:42.245967 10630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:42.245997 10630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:42.246021 10630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:42.247355 10630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:42867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=36219
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:42.247601 10630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:42.247776 10630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:42.252399 10774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.252429 10782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.252602 10630 server_base.cc:1034] running on GCE node
W20260809 19:30:42.252645 10778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.252852 10630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:42.253122 10630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:42.254276 10630 hybrid_clock.cc:648] HybridClock initialized: now 1786303842254251 us; error 51 us; skew 500 ppm
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:42.256776 10630 init.cc:377] Logged in from keytab as oryx/127.31.57.193@KRBTEST.COM (short username oryx)
I20260809 19:30:42.257210 10630 webserver.cc:460] Webserver started at http://127.31.57.193:36219/ using document root <none> and password file <none>
I20260809 19:30:42.257409 10630 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.257463 10630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.258693 10630 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:42.259271 10794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.259478 10630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.259577 10630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "381a04c485bf641b9ef1f4e6f59c68f3"
server_key_iv: "07c58caf7a1597737aecf6e26f250905"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.259948 10630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.275166 10630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:42.276922 10630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:42.277065 10630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:42.277262 10630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:42.277544 10630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:42.277590 10630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.277621 10630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:42.277645 10630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.283368 10630 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:42867
I20260809 19:30:42.283445 10980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:42867 every 8 connection(s)
I20260809 19:30:42.283736 10630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:42.286394 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 10630
I20260809 19:30:42.286545 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 9715
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:42.296952 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:38975 (local address 127.31.57.253:43035)
0809 19:30:42.284540 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.284542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.284913 (+ 371us) server_negotiation.cc:407] Connection header received
0809 19:30:42.285471 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.285473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.285485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.285534 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.286343 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287062 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.288794 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289087 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289127 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.293482 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.293483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.293484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.293492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.295437 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.295895 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.295896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.295897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.295944 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.296213 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.296214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.296214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.296365 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.296405 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.296782 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.296864 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:42.297142 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.31.57.193:56027)
0809 19:30:42.284805 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.284836 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.285362 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.285630 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.285635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.285976 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.286185 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.286191 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287448 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.287448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.288687 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.288690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289197 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.289340 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.290949 (+ 1609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.290953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.293782 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.295844 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.295845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.295845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.296035 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.296219 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.296220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.296220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.296265 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.296531 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.296531 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.296609 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.296735 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.296905 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":108,"thread_start_us":83,"threads_started":1}
I20260809 19:30:42.297345 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.31.57.193:38975)
0809 19:30:42.284803 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.284835 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.285369 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.285966 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.285967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.286119 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.286220 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.286221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287180 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.287180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.288687 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.288690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289200 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.289339 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.290885 (+ 1546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.290907 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.293360 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.295527 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.295535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.295538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.295816 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.296013 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.296014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.296014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.296136 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.296448 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.296455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.296609 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.297097 (+ 488us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.297233 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":129,"spinlock_wait_cycles":6912,"thread_start_us":84,"threads_started":1}
I20260809 19:30:42.297492 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:56027 (local address 127.31.57.252:40479)
0809 19:30:42.284995 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.284997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.285005 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:42.285519 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.285520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.285534 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.285601 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.286308 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287307 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.288798 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289088 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289140 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.293879 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.293880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.293880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.293888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.295754 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.296104 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.296105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.296105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.296155 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.296328 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.296329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.296329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.296458 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.296510 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.297305 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.297396 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":62,"threads_started":1}
I20260809 19:30:42.297785 10985 heartbeater.cc:347] Connected to a master server at 127.31.57.253:43035
I20260809 19:30:42.297878 10985 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.298036 10985 heartbeater.cc:511] Master 127.31.57.253:43035 requested a full tablet report, sending...
I20260809 19:30:42.298538 10389 ts_manager.cc:194] Registered new tserver with Master: e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867)
I20260809 19:30:42.299242 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:36595 (local address 127.31.57.254:33587)
0809 19:30:42.284488 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.284490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.284928 (+ 438us) server_negotiation.cc:407] Connection header received
0809 19:30:42.285490 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.285493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.285507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.285540 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.286328 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287149 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.288886 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289245 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289292 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.294769 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.294770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.294771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.294794 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.297223 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.298355 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.298356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.298357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.298432 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.298734 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.298735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.298735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.298875 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.298929 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.299124 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.299187 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:42.299649 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.284664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:36595)
0809 19:30:42.284865 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.284876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.285365 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.285632 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.285636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.285968 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.286184 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.286189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.287262 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.287263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.288744 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.288748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.289379 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.289380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.289454 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.291093 (+ 1639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.291099 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.294064 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.297870 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.297872 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.297872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.298239 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.298534 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.298535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.298535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.298610 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.298950 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.298951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.299007 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.299106 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.299230 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":58,"thread_start_us":65,"threads_started":1}
I20260809 19:30:42.300004 10984 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:42.300057 10984 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.300132 10984 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:42.300343 9378 ts_manager.cc:194] Re-registered known tserver with Master: e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867)
I20260809 19:30:42.300772 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:36595
I20260809 19:30:42.306059 10983 heartbeater.cc:347] Connected to a master server at 127.31.57.252:40479
I20260809 19:30:42.306129 10983 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.306201 10983 heartbeater.cc:511] Master 127.31.57.252:40479 requested a full tablet report, sending...
I20260809 19:30:42.306568 10690 ts_manager.cc:194] Registered new tserver with Master: e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867)
I20260809 19:30:42.309947 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:37039
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:33587
--webserver_port=37621
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:42.388190 10994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:42.388370 10994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:42.388393 10994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:42.389395 10994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:42.389424 10994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:42.389446 10994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:42.390828 10994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:37039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=37621
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:42.391070 10994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:42.391249 10994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:42.396111 11013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.396111 11011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.396180 10994 server_base.cc:1034] running on GCE node
W20260809 19:30:42.396117 11009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.396548 10994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:42.396740 10994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:42.397874 10994 hybrid_clock.cc:648] HybridClock initialized: now 1786303842397853 us; error 36 us; skew 500 ppm
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:42.400272 10994 init.cc:377] Logged in from keytab as oryx/127.31.57.194@KRBTEST.COM (short username oryx)
I20260809 19:30:42.400653 10994 webserver.cc:460] Webserver started at http://127.31.57.194:37621/ using document root <none> and password file <none>
I20260809 19:30:42.400884 10994 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.400952 10994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.402200 10994 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:42.402727 11025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.402887 10994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:42.402971 10994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "f475f17211444ea680c4a80e8e53803a"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "429424d261191e67e552acb57c6cb790"
server_key_iv: "a97474e6d72801b9f33066116918ceda"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.403266 10994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.425086 10994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:42.426653 10994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:42.426808 10994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:42.427021 10994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:42.427304 10994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:42.427336 10994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.427387 10994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:42.427449 10994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.432806 10994 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:37039
I20260809 19:30:42.432837 11202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:37039 every 8 connection(s)
I20260809 19:30:42.433198 10994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:42.434826 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 10994
I20260809 19:30:42.434970 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 9942
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
I20260809 19:30:42.451896 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:37301
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:33587
--webserver_port=43261
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:42.456384 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.434209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:56827 (local address 127.31.57.254:33587)
0809 19:30:42.434265 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.434268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.434406 (+ 138us) server_negotiation.cc:407] Connection header received
0809 19:30:42.446082 (+ 11676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.446085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.446099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.446135 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.447223 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.447991 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.449888 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.450521 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.450595 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.453253 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.453255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.453255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.453262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.454958 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455505 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455563 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455739 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455872 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.455919 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.456194 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.456274 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:30:42.456390 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.433975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:43455 (local address 127.31.57.253:43035)
0809 19:30:42.434027 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.434029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.434451 (+ 422us) server_negotiation.cc:407] Connection header received
0809 19:30:42.446979 (+ 12528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.446981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.446993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.447022 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.447483 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.448114 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.449447 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.449758 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.449831 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.452355 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.452356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.452357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.452364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.454938 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455485 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455535 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455716 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455821 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.455880 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.456248 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.456313 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:30:42.456480 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.433915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.31.57.194:47115)
0809 19:30:42.434632 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.434639 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.447228 (+ 12589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.447442 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.447443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.447618 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.447703 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.447705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.449456 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.449457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.449993 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.449995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.450531 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.450532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.450603 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.450983 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.450988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.452998 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.454902 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.454908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.454911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455277 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455497 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.455498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455612 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455873 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.455879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.456024 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.456115 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.456279 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":182,"thread_start_us":71,"threads_started":1}
I20260809 19:30:42.456728 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.433916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:56827)
0809 19:30:42.434297 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.434325 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.445931 (+ 11606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.446364 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.446370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.446862 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.447120 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.447126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.448100 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.448101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.449339 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.449341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.450661 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.450662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.450998 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.451424 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.451428 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.453157 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.455047 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.455048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455426 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455626 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.455627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455680 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455951 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.455952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.456026 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.456488 (+ 462us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.456594 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":58,"thread_start_us":85,"threads_started":1}
I20260809 19:30:42.456820 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.433916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.31.57.194:43455)
0809 19:30:42.434257 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.434327 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.446876 (+ 12549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.447102 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.447103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.447306 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.447388 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.447390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.448192 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.448192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.449339 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.449341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.449853 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.449854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.449981 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.450526 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.450536 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.452267 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.455040 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.455041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455415 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455595 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.455596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.455656 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.455900 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.455901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.456032 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.456115 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.456736 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":511,"thread_start_us":97,"threads_started":1}
I20260809 19:30:42.457098 11205 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:42.457180 11205 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.457278 11206 heartbeater.cc:347] Connected to a master server at 127.31.57.253:43035
I20260809 19:30:42.457319 11206 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.457365 11205 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:42.457667 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.434043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:47115 (local address 127.31.57.252:40479)
0809 19:30:42.434085 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.434087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.434728 (+ 641us) server_negotiation.cc:407] Connection header received
0809 19:30:42.447308 (+ 12580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.447309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.447322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.447349 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.447797 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.448473 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.450114 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.450406 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.450448 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.453102 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.453103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.453103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.453110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.454810 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455358 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455433 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.455675 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.455676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.455676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.455793 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.455848 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.457465 (+ 1617us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.457573 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:30:42.457384 11206 heartbeater.cc:511] Master 127.31.57.253:43035 requested a full tablet report, sending...
I20260809 19:30:42.457787 9378 ts_manager.cc:194] Re-registered known tserver with Master: f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
I20260809 19:30:42.457999 10389 ts_manager.cc:194] Registered new tserver with Master: f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
I20260809 19:30:42.458084 11204 heartbeater.cc:347] Connected to a master server at 127.31.57.252:40479
I20260809 19:30:42.458115 11204 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.458173 11204 heartbeater.cc:511] Master 127.31.57.252:40479 requested a full tablet report, sending...
I20260809 19:30:42.458199 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:56827
I20260809 19:30:42.458428 10690 ts_manager.cc:194] Registered new tserver with Master: f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
W20260809 19:30:42.529719 11216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:42.529892 11216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:42.529912 11216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:42.530872 11216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:42.530901 11216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:42.530923 11216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:42.532147 11216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:37301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=43261
--tserver_master_addrs=127.31.57.254:33587,127.31.57.253:43035,127.31.57.252:40479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:42.532305 11216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:42.532457 11216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:42.536912 11224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.536938 11223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:42.536948 11226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:42.537241 11216 server_base.cc:1034] running on GCE node
I20260809 19:30:42.537431 11216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:42.537597 11216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:42.538722 11216 hybrid_clock.cc:648] HybridClock initialized: now 1786303842538713 us; error 28 us; skew 500 ppm
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:42.540983 11216 init.cc:377] Logged in from keytab as oryx/127.31.57.195@KRBTEST.COM (short username oryx)
I20260809 19:30:42.541438 11216 webserver.cc:460] Webserver started at http://127.31.57.195:43261/ using document root <none> and password file <none>
I20260809 19:30:42.541594 11216 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:42.541637 11216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:42.542791 11216 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:42.543408 11239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.543599 11216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:30:42.543706 11216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "fa79d60dbe39406f81dba806f3431da5"
format_stamp: "Formatted at 2026-08-09 19:30:41 on dist-test-slave-x4qh"
server_key: "c980861121e191a7bdfb12000c95753f"
server_key_iv: "64b90e517a96587bb754e295e58e91d9"
server_key_version: "encryptionkey@0"
I20260809 19:30:42.543989 11216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:42.555469 11216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:42.557657 11216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:42.557801 11216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:42.558001 11216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:42.558279 11216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:42.558310 11216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.558379 11216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:42.558420 11216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:42.563997 11216 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:37301
I20260809 19:30:42.564064 11426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:37301 every 8 connection(s)
I20260809 19:30:42.564381 11216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:42.567785 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 11216
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.253@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.252@KRBTEST.COM
Aug 09 19:30:42 dist-test-slave-x4qh krb5kdc[9299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303842, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:42.578269 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:53579)
0809 19:30:42.566583 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.566592 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.566660 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.566840 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.566841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.567014 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.567104 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.567106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.568030 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.568031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569201 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.569204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569750 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569830 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.570524 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.570529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.573609 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.576707 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.576713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.576715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.576994 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.577211 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.577212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.577346 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.577665 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.577671 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.577808 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.577916 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.578056 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1533,"mutex_wait_us":129,"thread_start_us":119,"threads_started":1}
I20260809 19:30:42.578574 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.31.57.195:52443)
0809 19:30:42.566430 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.566458 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.566506 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.566637 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.566638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.566933 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.567102 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.567106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.567927 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.567928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569099 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.569102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569807 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569859 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.570460 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.570471 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.573609 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.577464 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.577466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.577693 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.577896 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.577897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.577952 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.578219 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.578220 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.578277 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.578338 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.578440 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1388,"mutex_wait_us":48,"thread_start_us":221,"threads_started":1}
I20260809 19:30:42.578812 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:53579 (local address 127.31.57.254:33587)
0809 19:30:42.565334 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.565336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.566636 (+ 1300us) server_negotiation.cc:407] Connection header received
0809 19:30:42.566730 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.566731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.566743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.566771 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.567207 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.567933 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569332 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569614 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569692 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.573750 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.573751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.573751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.573757 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.576643 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577077 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.577077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.577078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.577139 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577428 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.577430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.577430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.577572 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.577627 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.578614 (+ 987us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.578711 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 19:30:42.578883 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:52443 (local address 127.31.57.253:43035)
0809 19:30:42.565296 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.565298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.566516 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 19:30:42.566552 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.566553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.566563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.566616 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.567196 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.567793 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569211 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569497 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569551 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.573712 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.573714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.573714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.573721 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.577378 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.577768 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.577770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.577770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.577832 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.578011 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.578012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.578012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.578143 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.578196 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.578680 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.578746 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 19:30:42.579378 11429 heartbeater.cc:347] Connected to a master server at 127.31.57.253:43035
I20260809 19:30:42.579485 11429 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.579487 11428 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33587
I20260809 19:30:42.579571 11428 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.579571 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.569767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.31.57.252:40479)
0809 19:30:42.570264 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.570266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.570417 (+ 151us) server_negotiation.cc:407] Connection header received
0809 19:30:42.570439 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.570440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.570451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.570504 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.571320 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.572065 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.572985 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.573261 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.573300 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.576059 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.576060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.576060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.576067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.577894 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.578244 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.578245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.578246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.578324 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.578737 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.578738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.578739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.578888 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.578963 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.579341 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.579453 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":58,"threads_started":1}
I20260809 19:30:42.579668 11429 heartbeater.cc:511] Master 127.31.57.253:43035 requested a full tablet report, sending...
I20260809 19:30:42.579679 11428 heartbeater.cc:511] Master 127.31.57.254:33587 requested a full tablet report, sending...
I20260809 19:30:42.580067 9378 ts_manager.cc:194] Re-registered known tserver with Master: fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:42.580125 10389 ts_manager.cc:194] Registered new tserver with Master: fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:42.580554 9378 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:53579
I20260809 19:30:42.581952 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50349 (local address 127.31.57.252:40479)
0809 19:30:42.565348 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.565350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.565435 (+ 85us) server_negotiation.cc:407] Connection header received
0809 19:30:42.566393 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.566394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.566402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.566455 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.567205 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.567936 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569272 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569562 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569615 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.573772 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.573773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.573774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.573792 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.575986 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.580612 (+ 4626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.580613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.580613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.580672 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.581296 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.581297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.581298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.581434 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.581523 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.581762 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.581839 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:30:42.582002 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.570292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.31.57.253:43035)
0809 19:30:42.572528 (+ 2236us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.572529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.572536 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:42.572555 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.572556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.572567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.574050 (+ 1483us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.574084 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.574807 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.575882 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.576651 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.576928 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.578002 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.578002 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.578003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.578009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.580637 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.581009 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.581009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.581009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.581053 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.581251 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.581252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.581252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.581404 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.581788 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.581823 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.581888 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2216,"thread_start_us":57,"threads_started":1}
I20260809 19:30:42.582458 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.565020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.31.57.195:50349)
0809 19:30:42.565318 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.565361 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.566286 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.566552 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.566556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.566979 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:42.567104 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.567106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.568083 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.568084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569201 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.569204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.569687 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.569688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.569828 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.570653 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:42.570658 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:42.573679 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:42.580297 (+ 6618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.580298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.580298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.580543 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.580739 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:42.580740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:42.580740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:42.580818 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:42.581581 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:42.581582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:42.581625 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.582319 (+ 694us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.582395 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":143,"thread_start_us":180,"threads_started":1}
I20260809 19:30:42.583197 11427 heartbeater.cc:347] Connected to a master server at 127.31.57.252:40479
I20260809 19:30:42.583246 11427 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:42.583318 11427 heartbeater.cc:511] Master 127.31.57.252:40479 requested a full tablet report, sending...
I20260809 19:30:42.583639 10690 ts_manager.cc:194] Registered new tserver with Master: fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:42.584475 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.575226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.31.57.252:40479)
0809 19:30:42.577008 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.577010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.577017 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:42.577064 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.577065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.577092 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.581141 (+ 4049us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.581176 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.581883 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.583546 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.584112 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.584213 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.584305 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.584378 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.584414 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1759,"thread_start_us":62,"threads_started":1}
I20260809 19:30:42.584478 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.575024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35916)
0809 19:30:42.575112 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.575121 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.575155 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.578083 (+ 2928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.578084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.578086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.578119 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.578121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.581980 (+ 3859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.581981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.582886 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.582889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.584214 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.584215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.584296 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.584300 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.584372 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.584418 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 19:30:42.584757 10693 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260809 19:30:42.584885 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 112dfe4504574f149c2ab83d17e096b0. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: NON_VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: true } } }
I20260809 19:30:42.584991 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:30:42.585110 10328 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: attempting to promote NON_VOTER 1fa75af971344d12ad205379acc506b7 to VOTER
I20260809 19:30:42.585273 10328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } }
I20260809 19:30:42.585752 10693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 LEARNER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 19:30:42.586201 10405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Refusing update from remote peer 112dfe4504574f149c2ab83d17e096b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 19:30:42.586400 10615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 19:30:42.586711 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.569647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35072 (local address 127.31.57.254:33587)
0809 19:30:42.572359 (+ 2712us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.572361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.572368 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:42.572386 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.572387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.572397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.572462 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.573267 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.573953 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.576142 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.576432 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.576509 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.583028 (+ 6519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.583029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.583029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.583035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.584921 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.585339 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.585340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.585340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.585440 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.586022 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.586023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.586024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.586216 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.586304 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.586557 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.586638 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2687,"thread_start_us":89,"threads_started":1}
I20260809 19:30:42.586881 10615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 19:30:42.587459 10615 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1fa75af971344d12ad205379acc506b7 (127.31.57.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.587800 10693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1fa75af971344d12ad205379acc506b7 (127.31.57.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.587965 10405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1fa75af971344d12ad205379acc506b7 (127.31.57.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.588644 10615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.588732 10615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.588780 10615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.588819 10615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.588855 10615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.588887 10615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:42.588948 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.589010 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:42.589073 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "112dfe4504574f149c2ab83d17e096b0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:42.589145 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:42.589849 9378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:42.591300 9378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:42.598340 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.594808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:37301 (local address 127.0.0.1:49410)
0809 19:30:42.595181 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.595189 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.595225 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.595520 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.595521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.595522 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.595560 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.595562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596331 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.596332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.597437 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.597439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.598087 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.598088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.598173 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.598176 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.598225 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.598266 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":63,"thread_start_us":85,"threads_started":1}
I20260809 19:30:42.598451 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.594944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59336 (local address 127.31.57.194:37039)
0809 19:30:42.595544 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.595548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.595558 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:42.595592 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.595594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.595607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.595692 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.595768 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596603 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.597609 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.598073 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.598157 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.598230 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.598327 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.598373 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":76,"threads_started":1}
I20260809 19:30:42.598448 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.594733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37039 (local address 127.0.0.1:59336)
0809 19:30:42.595031 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.595041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.595104 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.595690 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.595691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.595692 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.595719 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.595720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596710 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.596711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.597545 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.597547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.598190 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.598191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.598248 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.598252 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.598311 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.598347 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":91,"threads_started":1}
I20260809 19:30:42.599231 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.595012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49410 (local address 127.31.57.195:37301)
0809 19:30:42.595329 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.595333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.595342 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:42.595436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.595438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.595452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.595524 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.595607 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596250 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.597526 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.597999 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.598061 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.598146 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.599150 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.599183 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
I20260809 19:30:42.599184 11093 tablet_service.cc:1467] Processing CreateTablet for tablet f0398212d417482aa4284d25de2dc946 (DEFAULT_TABLE table=test-table [id=7129003ad92a4dc8a9659855796d4953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:42.599318 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.594697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42867 (local address 127.0.0.1:52350)
0809 19:30:42.594744 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.594754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.594790 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.595119 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.595120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.595121 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.595163 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.595164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596357 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.596357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.597437 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.597439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.599093 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.599094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.599156 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.599159 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.599218 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.599255 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":45}
I20260809 19:30:42.599558 11093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0398212d417482aa4284d25de2dc946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:42.599830 11325 tablet_service.cc:1467] Processing CreateTablet for tablet f0398212d417482aa4284d25de2dc946 (DEFAULT_TABLE table=test-table [id=7129003ad92a4dc8a9659855796d4953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:42.600142 11325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0398212d417482aa4284d25de2dc946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:42.603163 11475 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Bootstrap starting.
I20260809 19:30:42.603682 11476 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Bootstrap starting.
I20260809 19:30:42.603874 11475 tablet_bootstrap.cc:654] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:42.604015 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.594797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.31.57.193:42867)
0809 19:30:42.594955 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.594959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.594968 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:42.595010 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.595012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.595028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.595122 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.595226 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.596262 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.598187 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.599023 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.599075 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.599117 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.603861 (+ 4744us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.603913 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260809 19:30:42.604202 11475 log.cc:824] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:42.604351 11476 tablet_bootstrap.cc:654] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:42.604669 11476 log.cc:824] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:42.604590 10876 tablet_service.cc:1467] Processing CreateTablet for tablet f0398212d417482aa4284d25de2dc946 (DEFAULT_TABLE table=test-table [id=7129003ad92a4dc8a9659855796d4953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:42.604910 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0398212d417482aa4284d25de2dc946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:42.605049 11475 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: No bootstrap required, opened a new log
I20260809 19:30:42.605111 11475 ts_tablet_manager.cc:1397] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:42.605413 11476 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: No bootstrap required, opened a new log
I20260809 19:30:42.605481 11476 ts_tablet_manager.cc:1397] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:42.607072 11476 raft_consensus.cc:348] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.607215 11476 raft_consensus.cc:374] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:42.607259 11476 raft_consensus.cc:729] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa79d60dbe39406f81dba806f3431da5, State: Initialized, Role: FOLLOWER
I20260809 19:30:42.607415 11476 consensus_queue.cc:260] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.607705 11476 ts_tablet_manager.cc:1428] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:42.607821 11428 heartbeater.cc:503] Master 127.31.57.254:33587 was elected leader, sending a full tablet report...
I20260809 19:30:42.607719 11475 raft_consensus.cc:348] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.607878 11475 raft_consensus.cc:374] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:42.608570 11475 raft_consensus.cc:729] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f475f17211444ea680c4a80e8e53803a, State: Initialized, Role: FOLLOWER
I20260809 19:30:42.608685 11475 consensus_queue.cc:260] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.608963 11475 ts_tablet_manager.cc:1428] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 19:30:42.609051 11205 heartbeater.cc:503] Master 127.31.57.254:33587 was elected leader, sending a full tablet report...
I20260809 19:30:42.609158 11481 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Bootstrap starting.
I20260809 19:30:42.610212 11481 tablet_bootstrap.cc:654] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:42.610514 11481 log.cc:824] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:42.611150 11481 tablet_bootstrap.cc:492] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: No bootstrap required, opened a new log
I20260809 19:30:42.611198 11481 ts_tablet_manager.cc:1397] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:42.612866 11481 raft_consensus.cc:348] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.612982 11481 raft_consensus.cc:374] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:42.613029 11481 raft_consensus.cc:729] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f621b82ec24502a4a21dd95cf4b4b9, State: Initialized, Role: FOLLOWER
I20260809 19:30:42.613132 11481 consensus_queue.cc:260] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.613359 11481 ts_tablet_manager.cc:1428] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:30:42.613415 10984 heartbeater.cc:503] Master 127.31.57.254:33587 was elected leader, sending a full tablet report...
I20260809 19:30:42.618414 11480 raft_consensus.cc:482] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:42.618479 11480 raft_consensus.cc:504] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.618754 11480 leader_election.cc:288] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867), f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
I20260809 19:30:42.621201 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.618958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:49161 (local address 127.31.57.193:42867)
0809 19:30:42.619010 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.619012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.619022 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:42.619096 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.619097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.619103 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.619170 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.619299 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.619764 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620553 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.620879 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620961 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.620969 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.621111 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.621143 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:30:42.621214 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.618958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54551 (local address 127.31.57.194:37039)
0809 19:30:42.619056 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.619058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.619144 (+ 86us) server_negotiation.cc:407] Connection header received
0809 19:30:42.619196 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.619197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.619206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.619256 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:42.619417 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.619796 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620563 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.620892 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620966 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.620973 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.621126 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.621159 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 19:30:42.621238 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.618891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37039 (local address 127.31.57.195:54551)
0809 19:30:42.619100 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.619107 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.619152 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.619259 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.619259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.619260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.619322 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.619323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.619864 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.619865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.620481 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.620483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.620965 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.621048 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.621051 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.621117 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.621154 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":23}
I20260809 19:30:42.621346 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.618891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42867 (local address 127.31.57.195:49161)
0809 19:30:42.618986 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:42.618997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:42.619034 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:42.619176 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:42.619177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:42.619179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:42.619233 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.619235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.619833 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.619834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.620470 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:42.620472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.620962 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.620963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:42.621038 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.621041 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:42.621103 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:30:42.621138 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 19:30:42.621522 11131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0398212d417482aa4284d25de2dc946" candidate_uuid: "fa79d60dbe39406f81dba806f3431da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f475f17211444ea680c4a80e8e53803a" is_pre_election: true
I20260809 19:30:42.621618 11131 raft_consensus.cc:2454] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa79d60dbe39406f81dba806f3431da5 in term 0.
I20260809 19:30:42.621665 10903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0398212d417482aa4284d25de2dc946" candidate_uuid: "fa79d60dbe39406f81dba806f3431da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" is_pre_election: true
I20260809 19:30:42.621764 10903 raft_consensus.cc:2454] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa79d60dbe39406f81dba806f3431da5 in term 0.
I20260809 19:30:42.621796 11243 leader_election.cc:302] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f475f17211444ea680c4a80e8e53803a, fa79d60dbe39406f81dba806f3431da5; no voters:
I20260809 19:30:42.621888 11480 raft_consensus.cc:2781] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:42.621953 11480 raft_consensus.cc:482] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:42.621984 11480 raft_consensus.cc:3037] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:42.622548 11480 raft_consensus.cc:504] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.622742 11480 leader_election.cc:288] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [CANDIDATE]: Term 1 election: Requested vote from peers e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867), f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039)
I20260809 19:30:42.622776 10903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0398212d417482aa4284d25de2dc946" candidate_uuid: "fa79d60dbe39406f81dba806f3431da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
I20260809 19:30:42.622830 10903 raft_consensus.cc:3037] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:42.622886 11131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0398212d417482aa4284d25de2dc946" candidate_uuid: "fa79d60dbe39406f81dba806f3431da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f475f17211444ea680c4a80e8e53803a"
I20260809 19:30:42.622948 11131 raft_consensus.cc:3037] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:42.623344 10903 raft_consensus.cc:2454] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa79d60dbe39406f81dba806f3431da5 in term 1.
I20260809 19:30:42.623534 11242 leader_election.cc:302] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7f621b82ec24502a4a21dd95cf4b4b9, fa79d60dbe39406f81dba806f3431da5; no voters:
I20260809 19:30:42.623570 11131 raft_consensus.cc:2454] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa79d60dbe39406f81dba806f3431da5 in term 1.
I20260809 19:30:42.623643 11480 raft_consensus.cc:2781] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:42.623781 11480 raft_consensus.cc:686] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 1 LEADER]: Becoming Leader. State: Replica: fa79d60dbe39406f81dba806f3431da5, State: Running, Role: LEADER
I20260809 19:30:42.623907 11480 consensus_queue.cc:237] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } }
I20260809 19:30:42.624569 9378 catalog_manager.cc:5495] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa79d60dbe39406f81dba806f3431da5 (127.31.57.195). New cstate: current_term: 1 leader_uuid: "fa79d60dbe39406f81dba806f3431da5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } health_report { overall_health: HEALTHY } } }
I20260809 19:30:42.632359 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.630534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.31.57.195:37301)
0809 19:30:42.630564 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.630566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.630575 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:42.630639 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.630640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.630657 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.630710 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:42.630805 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.631189 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.631853 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.632007 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.632055 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.632169 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:42.632235 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:42.632280 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.632309 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.632322 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:30:42.633643 10903 raft_consensus.cc:1261] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Refusing update from remote peer fa79d60dbe39406f81dba806f3431da5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:42.633915 11131 raft_consensus.cc:1261] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Refusing update from remote peer fa79d60dbe39406f81dba806f3431da5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:42.633941 11487 consensus_queue.cc:1035] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:42.634150 11480 consensus_queue.cc:1035] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:42.636421 11493 mvcc.cc:205] Tried to move back new op lower bound from 7316700539425525760 to 7316700539388112896. Current Snapshot: MvccSnapshot[applied={T|T < 7316700539425525760}]
I20260809 19:30:42.639269 9376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:42.639353 9376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:42.640534 9376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 112dfe4504574f149c2ab83d17e096b0: Sending DeleteTablet for 3 replicas of tablet f0398212d417482aa4284d25de2dc946
I20260809 19:30:42.640874 11093 tablet_service.cc:1514] Processing DeleteTablet for tablet f0398212d417482aa4284d25de2dc946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:42 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:59336
I20260809 19:30:42.641078 11509 tablet_replica.cc:330] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: stopping tablet replica
I20260809 19:30:42.641185 11509 raft_consensus.cc:2229] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:42.641274 11509 raft_consensus.cc:2258] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:42.641997 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 9319
I20260809 19:30:42.641997 10876 tablet_service.cc:1514] Processing DeleteTablet for tablet f0398212d417482aa4284d25de2dc946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:42 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:52350
I20260809 19:30:42.642197 11325 tablet_service.cc:1514] Processing DeleteTablet for tablet f0398212d417482aa4284d25de2dc946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:42 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:49410
I20260809 19:30:42.648171 11512 tablet_replica.cc:330] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: stopping tablet replica
I20260809 19:30:42.648273 11512 raft_consensus.cc:2229] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:42.648378 11512 raft_consensus.cc:2258] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:42.651923 11509 ts_tablet_manager.cc:1905] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:42.652280 11513 tablet_replica.cc:330] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: stopping tablet replica
I20260809 19:30:42.652392 11513 raft_consensus.cc:2229] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:42.652493 11513 raft_consensus.cc:2258] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:42.652951 11513 ts_tablet_manager.cc:1905] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:42.653781 11509 ts_tablet_manager.cc:1918] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:42.653885 11509 log.cc:1196] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/f0398212d417482aa4284d25de2dc946
I20260809 19:30:42.654217 11509 ts_tablet_manager.cc:1939] T f0398212d417482aa4284d25de2dc946 P f475f17211444ea680c4a80e8e53803a: Deleting consensus metadata
I20260809 19:30:42.654904 11512 ts_tablet_manager.cc:1905] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:42.654909 11513 ts_tablet_manager.cc:1918] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:42.654994 11513 log.cc:1196] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/f0398212d417482aa4284d25de2dc946
I20260809 19:30:42.655160 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.641739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.31.57.252:40479)
0809 19:30:42.641798 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.641799 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.641806 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:42.641823 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.641823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.641832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.641875 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.648078 (+ 6203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.648794 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.649582 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.649788 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.649869 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.651238 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.651240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.651240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.651247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.653334 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.653920 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.653922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.653922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.653985 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.654346 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.654347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.654348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.654489 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.654632 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.654875 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.655075 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:42.655283 11513 ts_tablet_manager.cc:1939] T f0398212d417482aa4284d25de2dc946 P fa79d60dbe39406f81dba806f3431da5: Deleting consensus metadata
W20260809 19:30:42.655436 11028 connection.cc:535] server connection from 127.0.0.1:59336 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:30:42.656522 11512 ts_tablet_manager.cc:1918] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:42.656603 11512 log.cc:1196] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/f0398212d417482aa4284d25de2dc946
I20260809 19:30:42.656873 11512 ts_tablet_manager.cc:1939] T f0398212d417482aa4284d25de2dc946 P e7f621b82ec24502a4a21dd95cf4b4b9: Deleting consensus metadata
W20260809 19:30:42.657181 10800 connection.cc:441] server connection from 127.0.0.1:52350 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52350 (request call id 1) could send its response
I20260809 19:30:42.663494 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.657387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.31.57.252:40479)
0809 19:30:42.657422 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.657424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.657764 (+ 340us) server_negotiation.cc:407] Connection header received
0809 19:30:42.657824 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.657825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.657834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.657884 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.658367 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.658789 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.659472 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.659741 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.659808 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.660914 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.660915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.660915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.660921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.662328 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.662695 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.662696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.662696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.662750 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.662966 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.662966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.662967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.663067 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.663161 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.663251 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.663327 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:42.665992 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:42.657516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.31.57.253:43035)
0809 19:30:42.657564 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:42.657565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:42.659532 (+ 1967us) server_negotiation.cc:407] Connection header received
0809 19:30:42.659598 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:42.659599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:42.659611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:42.659653 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:42.660299 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.660970 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.661904 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:42.662214 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:42.662292 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:42.663650 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:42.663660 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:42.663661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:42.663667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:42.665075 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.665364 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.665365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.665365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.665420 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:42.665569 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:42.665569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:42.665570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:42.665685 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:42.665765 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:42.665839 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 19:30:42.665911 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:43.637466 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:43.637204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:60639)
0809 19:30:43.637329 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:43.637407 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260809 19:30:43.637681 11205 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:43.638211 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:43.638000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:48383)
0809 19:30:43.638094 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:43.638165 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":55,"threads_started":1}
W20260809 19:30:43.638384 11428 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:43.638933 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:43.638731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:42343)
0809 19:30:43.638821 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:43.638897 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":44,"threads_started":1}
W20260809 19:30:43.639164 10984 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:44.141021 11531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 112dfe4504574f149c2ab83d17e096b0)
I20260809 19:30:44.141179 11531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } }
I20260809 19:30:44.141425 11531 leader_election.cc:288] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587), feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253:43035)
I20260809 19:30:44.141597 11533 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 112dfe4504574f149c2ab83d17e096b0)
I20260809 19:30:44.141736 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.141505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35528)
0809 19:30:44.141659 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.141699 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":58,"threads_started":1}
I20260809 19:30:44.141716 11533 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } }
W20260809 19:30:44.141847 10663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:44.141955 11533 leader_election.cc:288] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587), 1fa75af971344d12ad205379acc506b7 (127.31.57.252:40479)
I20260809 19:30:44.142257 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.142005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35538)
0809 19:30:44.142163 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.142218 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":38,"threads_started":1}
I20260809 19:30:44.142263 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.142139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35554)
0809 19:30:44.142207 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.142234 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 19:30:44.142338 10663 leader_election.cc:334] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587): Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
W20260809 19:30:44.142474 10366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:44.142910 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.142808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35556)
0809 19:30:44.142856 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.142885 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 19:30:44.142999 10366 leader_election.cc:334] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587): Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:44.144937 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.141563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:43035 (local address 127.0.0.1:37786)
0809 19:30:44.141719 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.141732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.141820 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.142014 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.142015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.142018 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.142079 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.142081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.142917 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.142918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.144185 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.144188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.144757 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.144758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.144808 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.144812 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.144850 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.144885 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":48,"threads_started":1}
I20260809 19:30:44.144963 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.141616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.31.57.253:43035)
0809 19:30:44.141752 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.141754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.141869 (+ 115us) server_negotiation.cc:407] Connection header received
0809 19:30:44.141895 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.141896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.141907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.141960 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.142151 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.142849 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.144264 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.144674 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.144739 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.144750 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.144867 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.144905 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":42,"threads_started":1}
I20260809 19:30:44.145205 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1fa75af971344d12ad205379acc506b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" is_pre_election: true
I20260809 19:30:44.145309 10405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa75af971344d12ad205379acc506b7 in term 1.
I20260809 19:30:44.145460 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.142281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.31.57.252:40479)
0809 19:30:44.142433 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.142435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.142442 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:44.142525 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.142526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.142537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.142613 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.142745 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.143338 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.144599 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.145110 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.145205 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.145213 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.145371 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.145411 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260809 19:30:44.145490 10667 leader_election.cc:302] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fa75af971344d12ad205379acc506b7, feb2712f97384bb3a9265c4bb59ee1c4; no voters: 112dfe4504574f149c2ab83d17e096b0
I20260809 19:30:44.145598 11531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:30:44.145635 11531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 112dfe4504574f149c2ab83d17e096b0)
I20260809 19:30:44.145646 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.142118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40479 (local address 127.0.0.1:35950)
0809 19:30:44.142389 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.142400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.142454 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.142615 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.142616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.142617 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.142661 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.142663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.143484 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.143485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.144529 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.144531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.145208 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.145209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.145275 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.145278 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.145539 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.145591 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260809 19:30:44.145655 11531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:30:44.145841 10693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1fa75af971344d12ad205379acc506b7" is_pre_election: true
I20260809 19:30:44.146445 11531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } }
I20260809 19:30:44.146592 10693 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate feb2712f97384bb3a9265c4bb59ee1c4 in current term 2: Already voted for candidate 1fa75af971344d12ad205379acc506b7 in this term.
I20260809 19:30:44.146605 11531 leader_election.cc:288] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 election: Requested vote from peers 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587), feb2712f97384bb3a9265c4bb59ee1c4 (127.31.57.253:43035)
I20260809 19:30:44.146762 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1fa75af971344d12ad205379acc506b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "feb2712f97384bb3a9265c4bb59ee1c4"
I20260809 19:30:44.146791 10367 leader_election.cc:302] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: feb2712f97384bb3a9265c4bb59ee1c4; no voters: 112dfe4504574f149c2ab83d17e096b0, 1fa75af971344d12ad205379acc506b7
I20260809 19:30:44.146853 10405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:30:44.146955 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.146670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35566)
0809 19:30:44.146898 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.146929 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260809 19:30:44.147574 10405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa75af971344d12ad205379acc506b7 in term 2.
I20260809 19:30:44.147687 11533 raft_consensus.cc:2726] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:30:44.147713 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.147602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35570)
0809 19:30:44.147657 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.147689 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:30:44.147737 10667 leader_election.cc:302] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa75af971344d12ad205379acc506b7, feb2712f97384bb3a9265c4bb59ee1c4; no voters:
W20260809 19:30:44.147790 10663 leader_election.cc:334] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587): Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
I20260809 19:30:44.147821 11531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:30:44.147965 11531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [term 2 LEADER]: Becoming Leader. State: Replica: 1fa75af971344d12ad205379acc506b7, State: Running, Role: LEADER
I20260809 19:30:44.148038 11531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } }
I20260809 19:30:44.148389 11547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fa75af971344d12ad205379acc506b7. Latest consensus state: current_term: 2 leader_uuid: "1fa75af971344d12ad205379acc506b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:44.148466 11547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:44.148631 11549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:44.148994 11549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:44.149077 11549 catalog_manager.cc:1259] Loaded cluster ID: 6d5f082248a24ebd91958733f0d2d350
I20260809 19:30:44.149107 11549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:44.149247 11549 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:44.149334 11549 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Loaded TSK: 0
I20260809 19:30:44.149459 11549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:30:44.201647 10690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:44.203037 10690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:44.203941 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.203848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35584)
0809 19:30:44.203890 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.203915 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:30:44.203938 10405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [term 2 FOLLOWER]: Refusing update from remote peer 1fa75af971344d12ad205379acc506b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 19:30:44.204270 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.204141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.0.0.1:35600)
0809 19:30:44.204202 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.204225 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 19:30:44.204218 11547 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 19:30:44.204355 10663 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 -> Peer 112dfe4504574f149c2ab83d17e096b0 (127.31.57.254:33587): Couldn't send request to peer 112dfe4504574f149c2ab83d17e096b0. Status: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:30:44.204875 11533 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fa75af971344d12ad205379acc506b7. Latest consensus state: current_term: 2 leader_uuid: "1fa75af971344d12ad205379acc506b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:44.204957 11533 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:44.205384 11531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "1fa75af971344d12ad205379acc506b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:44.205479 11531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:44.205569 11551 mvcc.cc:205] Tried to move back new op lower bound from 7316700545857703936 to 7316700545630871552. Current Snapshot: MvccSnapshot[applied={T|T < 7316700545857703936}]
I20260809 19:30:44.205616 11533 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "1fa75af971344d12ad205379acc506b7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "112dfe4504574f149c2ab83d17e096b0" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33587 } } peers { permanent_uuid: "feb2712f97384bb3a9265c4bb59ee1c4" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 43035 } attrs { promote: false } } peers { permanent_uuid: "1fa75af971344d12ad205379acc506b7" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40479 } attrs { promote: false } } }
I20260809 19:30:44.205684 11533 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb2712f97384bb3a9265c4bb59ee1c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:44.209648 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42867 (local address 127.0.0.1:52358)
0809 19:30:44.207148 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.207158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.207196 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.207420 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.207421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.207423 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.207463 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.207464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.208156 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.208157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.208928 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.208930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.209443 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.209444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.209496 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.209500 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.209560 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.209597 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":2}
I20260809 19:30:44.209668 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52358 (local address 127.31.57.193:42867)
0809 19:30:44.207279 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.207281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.207289 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:44.207315 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.207316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.207327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.207417 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.207536 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.208087 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.209007 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.209351 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.209418 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.209429 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.209569 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.209608 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260809 19:30:44.210008 10876 tablet_service.cc:1467] Processing CreateTablet for tablet d0c8f49592264f4e807d42157ef26816 (DEFAULT_TABLE table=test-table [id=19996f46b5a04befbfcfca21e3e4140b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:44.210141 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c8f49592264f4e807d42157ef26816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:44.210810 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:37301 (local address 127.0.0.1:49422)
0809 19:30:44.207486 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.208164 (+ 678us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.208211 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.208338 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.208339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.208341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.208370 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.208371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.209144 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.209145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.210101 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.210103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.210601 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.210602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.210665 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.210669 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.210718 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.210755 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":74,"threads_started":1}
I20260809 19:30:44.210836 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.31.57.195:37301)
0809 19:30:44.207497 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.207499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.208224 (+ 725us) server_negotiation.cc:407] Connection header received
0809 19:30:44.208254 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.208255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.208266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.208332 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.208417 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.209039 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.210173 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.210526 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.210600 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.210611 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.210726 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.210768 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260809 19:30:44.210999 11325 tablet_service.cc:1467] Processing CreateTablet for tablet d0c8f49592264f4e807d42157ef26816 (DEFAULT_TABLE table=test-table [id=19996f46b5a04befbfcfca21e3e4140b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:44.211129 11325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c8f49592264f4e807d42157ef26816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:44.211633 11564 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: Bootstrap starting.
I20260809 19:30:44.211912 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37039 (local address 127.0.0.1:59350)
0809 19:30:44.207225 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.207233 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.207267 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.207686 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.207688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.207691 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.207739 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.207740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.208285 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.208286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.209211 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.209213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.211668 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.211669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.211726 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.211729 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.211773 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.211842 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 19:30:44.212064 11564 tablet_bootstrap.cc:654] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:44.212631 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.207338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.31.57.194:37039)
0809 19:30:44.207563 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.207565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.207572 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:44.207597 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.207598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.207607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.207669 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.207804 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.208185 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.209293 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.211578 (+ 2285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.212451 (+ 873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.212467 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.212486 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.212547 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":73,"threads_started":1}
I20260809 19:30:44.212684 11564 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: No bootstrap required, opened a new log
I20260809 19:30:44.212725 11564 ts_tablet_manager.cc:1397] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:44.212841 11564 raft_consensus.cc:348] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.212906 11564 raft_consensus.cc:374] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:44.212926 11564 raft_consensus.cc:729] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f621b82ec24502a4a21dd95cf4b4b9, State: Initialized, Role: FOLLOWER
I20260809 19:30:44.212971 11564 consensus_queue.cc:260] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.213119 11564 ts_tablet_manager.cc:1428] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:44.213258 11093 tablet_service.cc:1467] Processing CreateTablet for tablet d0c8f49592264f4e807d42157ef26816 (DEFAULT_TABLE table=test-table [id=19996f46b5a04befbfcfca21e3e4140b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:44.213390 11093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c8f49592264f4e807d42157ef26816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:44.213486 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.213290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:34255)
0809 19:30:44.213415 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.213451 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":90,"threads_started":1}
I20260809 19:30:44.213809 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.213713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:44047)
0809 19:30:44.213750 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.213773 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:30:44.213892 10983 heartbeater.cc:503] Master 127.31.57.252:40479 was elected leader, sending a full tablet report...
I20260809 19:30:44.214250 11568 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Bootstrap starting.
I20260809 19:30:44.214605 11568 tablet_bootstrap.cc:654] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:44.215088 11575 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Bootstrap starting.
I20260809 19:30:44.215266 11568 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: No bootstrap required, opened a new log
I20260809 19:30:44.215324 11568 ts_tablet_manager.cc:1397] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:44.215483 11568 raft_consensus.cc:348] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.215534 11568 raft_consensus.cc:374] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:44.215546 11568 raft_consensus.cc:729] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa79d60dbe39406f81dba806f3431da5, State: Initialized, Role: FOLLOWER
I20260809 19:30:44.215572 11575 tablet_bootstrap.cc:654] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:44.215595 11568 consensus_queue.cc:260] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.215732 11568 ts_tablet_manager.cc:1428] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:44.216079 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.215848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:46629)
0809 19:30:44.215964 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.216032 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":77,"threads_started":1}
I20260809 19:30:44.216178 11427 heartbeater.cc:503] Master 127.31.57.252:40479 was elected leader, sending a full tablet report...
I20260809 19:30:44.216396 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.216320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:49287)
0809 19:30:44.216352 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.216378 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 19:30:44.216753 11575 tablet_bootstrap.cc:492] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: No bootstrap required, opened a new log
I20260809 19:30:44.216804 11575 ts_tablet_manager.cc:1397] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:44.216936 11575 raft_consensus.cc:348] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.216991 11575 raft_consensus.cc:374] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:44.217003 11575 raft_consensus.cc:729] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f475f17211444ea680c4a80e8e53803a, State: Initialized, Role: FOLLOWER
I20260809 19:30:44.217077 11575 consensus_queue.cc:260] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.217227 11575 ts_tablet_manager.cc:1428] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:44.217582 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.217396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:54711)
0809 19:30:44.217520 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.217551 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260809 19:30:44.217669 11204 heartbeater.cc:503] Master 127.31.57.252:40479 was elected leader, sending a full tablet report...
I20260809 19:30:44.217944 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.217768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:57231)
0809 19:30:44.217795 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.217906 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 19:30:44.285392 10986 tablet.cc:2369] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:44.316494 11430 tablet.cc:2369] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:44.435825 11209 tablet.cc:2369] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:44.542110 11582 raft_consensus.cc:482] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:44.542191 11582 raft_consensus.cc:504] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.542495 11582 leader_election.cc:288] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867), fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:44.546339 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.542826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:57085 (local address 127.31.57.195:37301)
0809 19:30:44.542867 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.542869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.542966 (+ 97us) server_negotiation.cc:407] Connection header received
0809 19:30:44.543014 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.543014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.543021 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.543079 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.543170 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.543894 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.545452 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.546015 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.546070 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.546078 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.546231 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.546269 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:44.546339 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.542697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42779 (local address 127.31.57.193:42867)
0809 19:30:44.542756 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.542759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.542812 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:30:44.542843 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.542844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.542855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.542909 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:44.543097 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.543882 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.545450 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.545991 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.546067 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.546078 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.546231 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.546269 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:30:44.546375 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.542627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42867 (local address 127.31.57.194:42779)
0809 19:30:44.542682 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.542694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.542749 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.542931 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.542932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.542935 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.543024 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.543026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.543966 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.543967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.545362 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.545366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.546074 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.546075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.546154 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.546160 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.546216 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.546261 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":50}
I20260809 19:30:44.546380 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.542747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:37301 (local address 127.31.57.194:57085)
0809 19:30:44.542883 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.542911 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:44.542963 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:44.543080 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:44.543081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:44.543082 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:44.543129 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.543130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.543966 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.543967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.545363 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:44.545366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.546106 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.546107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:44.546163 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.546166 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:44.546216 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:30:44.546261 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":99,"thread_start_us":98,"threads_started":1}
I20260809 19:30:44.546630 10903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c8f49592264f4e807d42157ef26816" candidate_uuid: "f475f17211444ea680c4a80e8e53803a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" is_pre_election: true
I20260809 19:30:44.546725 10903 raft_consensus.cc:2454] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f475f17211444ea680c4a80e8e53803a in term 0.
I20260809 19:30:44.546846 11353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c8f49592264f4e807d42157ef26816" candidate_uuid: "f475f17211444ea680c4a80e8e53803a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa79d60dbe39406f81dba806f3431da5" is_pre_election: true
I20260809 19:30:44.546907 11028 leader_election.cc:302] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7f621b82ec24502a4a21dd95cf4b4b9, f475f17211444ea680c4a80e8e53803a; no voters:
I20260809 19:30:44.546967 11353 raft_consensus.cc:2454] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f475f17211444ea680c4a80e8e53803a in term 0.
I20260809 19:30:44.547044 11582 raft_consensus.cc:2781] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:44.547083 11582 raft_consensus.cc:482] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:44.547106 11582 raft_consensus.cc:3037] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:44.547871 11582 raft_consensus.cc:504] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.547966 11582 leader_election.cc:288] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [CANDIDATE]: Term 1 election: Requested vote from peers e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867), fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301)
I20260809 19:30:44.548120 10903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c8f49592264f4e807d42157ef26816" candidate_uuid: "f475f17211444ea680c4a80e8e53803a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9"
I20260809 19:30:44.548139 11353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c8f49592264f4e807d42157ef26816" candidate_uuid: "f475f17211444ea680c4a80e8e53803a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa79d60dbe39406f81dba806f3431da5"
I20260809 19:30:44.548202 11353 raft_consensus.cc:3037] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:44.548209 10903 raft_consensus.cc:3037] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:44.548806 10903 raft_consensus.cc:2454] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f475f17211444ea680c4a80e8e53803a in term 1.
I20260809 19:30:44.548854 11353 raft_consensus.cc:2454] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f475f17211444ea680c4a80e8e53803a in term 1.
I20260809 19:30:44.548995 11028 leader_election.cc:302] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7f621b82ec24502a4a21dd95cf4b4b9, f475f17211444ea680c4a80e8e53803a; no voters:
I20260809 19:30:44.549088 11582 raft_consensus.cc:2781] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:44.549237 11582 raft_consensus.cc:686] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 1 LEADER]: Becoming Leader. State: Replica: f475f17211444ea680c4a80e8e53803a, State: Running, Role: LEADER
I20260809 19:30:44.549319 11582 consensus_queue.cc:237] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } }
I20260809 19:30:44.549582 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.549506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:50705)
0809 19:30:44.549535 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.549559 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 19:30:44.549978 10690 catalog_manager.cc:5495] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a reported cstate change: term changed from 0 to 1, leader changed from <none> to f475f17211444ea680c4a80e8e53803a (127.31.57.194). New cstate: current_term: 1 leader_uuid: "f475f17211444ea680c4a80e8e53803a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f475f17211444ea680c4a80e8e53803a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37039 } health_report { overall_health: HEALTHY } } }
I20260809 19:30:44.560230 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.558339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59356 (local address 127.31.57.194:37039)
0809 19:30:44.558379 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:44.558380 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:44.558409 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:30:44.558438 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:44.558438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:44.558451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:44.558501 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:44.558602 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.558990 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.559678 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:44.559840 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:44.559887 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:44.559995 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:44.560078 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:44.560126 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:44.560158 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 19:30:44.560173 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:44.561126 10903 raft_consensus.cc:1261] T d0c8f49592264f4e807d42157ef26816 P e7f621b82ec24502a4a21dd95cf4b4b9 [term 1 FOLLOWER]: Refusing update from remote peer f475f17211444ea680c4a80e8e53803a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:44.561177 11353 raft_consensus.cc:1261] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Refusing update from remote peer f475f17211444ea680c4a80e8e53803a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:44.561367 11585 consensus_queue.cc:1035] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f621b82ec24502a4a21dd95cf4b4b9" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:44.561498 11582 consensus_queue.cc:1035] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa79d60dbe39406f81dba806f3431da5" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 37301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:44.562186 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.562083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:34177)
0809 19:30:44.562123 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.562160 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:30:44.562794 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.562575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:56849)
0809 19:30:44.562726 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.562758 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 19:30:44.563446 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.563321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.195:49643)
0809 19:30:44.563355 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.563411 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 19:30:44.564260 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.564121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:33709)
0809 19:30:44.564213 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.564236 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 19:30:44.565172 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.565070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.193:43107)
0809 19:30:44.565109 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.565133 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:30:44.565455 10688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:44.565552 10688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:44.565549 11589 mvcc.cc:205] Tried to move back new op lower bound from 7316700547320578048 to 7316700547274588160. Current Snapshot: MvccSnapshot[applied={T|T < 7316700547320578048 or (T in {7316700547320578048})}]
I20260809 19:30:44.566294 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.566195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33587 (local address 127.31.57.194:35461)
0809 19:30:44.566241 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.566271 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:30:44.566838 10688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa75af971344d12ad205379acc506b7: Sending DeleteTablet for 3 replicas of tablet d0c8f49592264f4e807d42157ef26816
I20260809 19:30:44.567072 11093 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c8f49592264f4e807d42157ef26816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:44 UTC) from {username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:59350
I20260809 19:30:44.567111 11091 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c8f49592264f4e807d42157ef26816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:44 UTC) from {username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:59350
I20260809 19:30:44.567253 11606 tablet_replica.cc:330] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: stopping tablet replica
I20260809 19:30:44.567305 11606 raft_consensus.cc:2229] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:44.567327 11325 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c8f49592264f4e807d42157ef26816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:44 UTC) from {username='oryx', principal='oryx/127.31.57.252@KRBTEST.COM'} at 127.0.0.1:49422
I20260809 19:30:44.567421 11606 raft_consensus.cc:2258] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:44.567426 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 10630
I20260809 19:30:44.572160 11606 ts_tablet_manager.cc:1905] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:44.573858 11606 ts_tablet_manager.cc:1918] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:44.573901 11606 log.cc:1196] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/d0c8f49592264f4e807d42157ef26816
I20260809 19:30:44.574127 11606 ts_tablet_manager.cc:1939] T d0c8f49592264f4e807d42157ef26816 P f475f17211444ea680c4a80e8e53803a: Deleting consensus metadata
I20260809 19:30:44.574383 10664 catalog_manager.cc:4841] TS f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039): tablet d0c8f49592264f4e807d42157ef26816 (table test-table [id=19996f46b5a04befbfcfca21e3e4140b]) successfully deleted
W20260809 19:30:44.574610 10664 catalog_manager.cc:4808] TS f475f17211444ea680c4a80e8e53803a (127.31.57.194:37039): delete failed for tablet d0c8f49592264f4e807d42157ef26816 because the tablet was not found. No further retry: Not found: Tablet not found: d0c8f49592264f4e807d42157ef26816
I20260809 19:30:44.587472 11608 tablet_replica.cc:330] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: stopping tablet replica
I20260809 19:30:44.587551 11608 raft_consensus.cc:2229] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:44.587605 11608 raft_consensus.cc:2258] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:44.587893 11608 ts_tablet_manager.cc:1905] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:44.589464 11608 ts_tablet_manager.cc:1918] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:44.589529 11608 log.cc:1196] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/d0c8f49592264f4e807d42157ef26816
I20260809 19:30:44.589772 11608 ts_tablet_manager.cc:1939] T d0c8f49592264f4e807d42157ef26816 P fa79d60dbe39406f81dba806f3431da5: Deleting consensus metadata
I20260809 19:30:44.590081 10663 catalog_manager.cc:4841] TS fa79d60dbe39406f81dba806f3431da5 (127.31.57.195:37301): tablet d0c8f49592264f4e807d42157ef26816 (table test-table [id=19996f46b5a04befbfcfca21e3e4140b]) successfully deleted
W20260809 19:30:44.591557 10663 connection.cc:535] client connection to 127.31.57.193:42867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:30:44.591979 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:44.591877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42867 (local address 127.0.0.1:52364)
0809 19:30:44.591918 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:44.591947 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:42867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 19:30:44.592097 10663 catalog_manager.cc:4584] TS e7f621b82ec24502a4a21dd95cf4b4b9 (127.31.57.193:42867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0c8f49592264f4e807d42157ef26816: Network error: Client connection negotiation failed: client connection to 127.31.57.193:42867: connect: Connection refused (error 111)
I20260809 19:30:44.592336 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 10994
I20260809 19:30:44.599519 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 11216
I20260809 19:30:44.609158 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 10163
I20260809 19:30:44.621255 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 10476
2026-08-09T19:30:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6259 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:44 dist-test-slave-x4qh krb5kdc[11615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:44 dist-test-slave-x4qh krb5kdc[11615](info): set up 2 sockets
Aug 09 19:30:44 dist-test-slave-x4qh krb5kdc[11615](info): commencing operation
krb5kdc: starting...
W20260809 19:30:45.763729 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:45 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab.
Entry for principal oryx/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab.
I20260809 19:30:45.870568 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34769
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab
--principal=oryx/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:45.946364 11636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:45.946606 11636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:45.946647 11636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:45.947719 11636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:45.947777 11636 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:30:45.947798 11636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:45.947822 11636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:45.947865 11636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:45.947896 11636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:45.949308 11636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34769
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.254.keytab
--principal=oryx/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:45.949546 11636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:45.949743 11636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:45.953962 11643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:45.953970 11642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:45.954002 11647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:45.954151 11636 server_base.cc:1034] running on GCE node
I20260809 19:30:45.954460 11636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:45.954746 11636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:45.955902 11636 hybrid_clock.cc:648] HybridClock initialized: now 1786303845955885 us; error 44 us; skew 500 ppm
Aug 09 19:30:45 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303845, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:45.958058 11636 init.cc:377] Logged in from keytab as oryx/127.31.57.254@KRBTEST.COM (short username oryx)
I20260809 19:30:45.958420 11636 webserver.cc:460] Webserver started at http://127.31.57.254:34277/ using document root <none> and password file <none>
I20260809 19:30:45.958628 11636 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:45.958689 11636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:45.958822 11636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:45.959765 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "3be1cae70fba45eabfc5cdad517cfa27"
format_stamp: "Formatted at 2026-08-09 19:30:45 on dist-test-slave-x4qh"
server_key: "651fa1659cc3d479dab2ca738edbf5ae"
server_key_iv: "10a10b1e16c6821d23eb081d184be76d"
server_key_version: "encryptionkey@0"
I20260809 19:30:45.960103 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "3be1cae70fba45eabfc5cdad517cfa27"
format_stamp: "Formatted at 2026-08-09 19:30:45 on dist-test-slave-x4qh"
server_key: "651fa1659cc3d479dab2ca738edbf5ae"
server_key_iv: "10a10b1e16c6821d23eb081d184be76d"
server_key_version: "encryptionkey@0"
I20260809 19:30:45.961285 11636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:45.961937 11657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:45.962087 11636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:45.962189 11636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "3be1cae70fba45eabfc5cdad517cfa27"
format_stamp: "Formatted at 2026-08-09 19:30:45 on dist-test-slave-x4qh"
server_key: "651fa1659cc3d479dab2ca738edbf5ae"
server_key_iv: "10a10b1e16c6821d23eb081d184be76d"
server_key_version: "encryptionkey@0"
I20260809 19:30:45.962249 11636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:45.983299 11636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:45.985096 11636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:45.985244 11636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:45.989754 11636 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34769
I20260809 19:30:45.989842 11761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34769 every 8 connection(s)
I20260809 19:30:45.990115 11636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:45.990805 11764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:45.992918 11764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Bootstrap starting.
I20260809 19:30:45.993467 11764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:45.993747 11764 log.cc:824] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:45.994377 11764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: No bootstrap required, opened a new log
I20260809 19:30:45.994967 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 11636
I20260809 19:30:45.995080 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:45.995282 31975 external_mini_cluster.cc:1428] Setting key 4f358b4fb6e9fe53f098e059a4f1df84
I20260809 19:30:45.996949 11764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } }
I20260809 19:30:45.997123 11764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:45.997161 11764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be1cae70fba45eabfc5cdad517cfa27, State: Initialized, Role: FOLLOWER
I20260809 19:30:45.997326 11764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } }
I20260809 19:30:45.997391 11764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:45.997411 11764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:45.997474 11764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:45.998036 11764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } }
I20260809 19:30:45.998138 11764 leader_election.cc:302] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be1cae70fba45eabfc5cdad517cfa27; no voters:
I20260809 19:30:45.998293 11764 leader_election.cc:288] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:45.998358 11774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:45.998531 11774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [term 1 LEADER]: Becoming Leader. State: Replica: 3be1cae70fba45eabfc5cdad517cfa27, State: Running, Role: LEADER
I20260809 19:30:45.998529 11764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:45.999621 11774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } }
I20260809 19:30:46.000321 11775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be1cae70fba45eabfc5cdad517cfa27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } } }
I20260809 19:30:46.000291 11776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be1cae70fba45eabfc5cdad517cfa27. Latest consensus state: current_term: 1 leader_uuid: "3be1cae70fba45eabfc5cdad517cfa27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be1cae70fba45eabfc5cdad517cfa27" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34769 } } }
I20260809 19:30:46.000420 11775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:46.000427 11776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:46.000643 11794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:46.001171 11794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:30:45 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.006865 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:45.995808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.31.57.254:34769)
0809 19:30:45.996022 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:45.996032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:45.996070 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:30:45.996908 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:45.996917 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:45.996972 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:45.997127 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:45.997846 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:45.999061 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.000001 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.000626 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.000745 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.003283 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.003287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.003287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.003297 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.005430 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.005805 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.005808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.005809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.005914 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.006121 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.006122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.006122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.006339 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.006510 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.006583 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.006708 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260809 19:30:46.007179 11794 catalog_manager.cc:1347] Generated new cluster ID: 1c4d1e5b7d1a45b2ae44ef071ef78200
I20260809 19:30:46.007239 11794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:46.027237 11794 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:46.027864 11794 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:46.036927 11794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Generated new TSK 0
I20260809 19:30:46.037077 11794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
I20260809 19:30:46.068082 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34769
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:46.142753 11807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:46.142966 11807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:46.143002 11807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:46.144039 11807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:46.144093 11807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:46.144124 11807 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:46.144157 11807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:46.145596 11807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:46.145819 11807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:46.146008 11807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:46.150518 11817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:46.150552 11815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:46.150538 11812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:46.150645 11807 server_base.cc:1034] running on GCE node
I20260809 19:30:46.150870 11807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:46.151080 11807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:46.152208 11807 hybrid_clock.cc:648] HybridClock initialized: now 1786303846152196 us; error 26 us; skew 500 ppm
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:46.154476 11807 init.cc:377] Logged in from keytab as oryx/127.31.57.193@KRBTEST.COM (short username oryx)
I20260809 19:30:46.154881 11807 webserver.cc:460] Webserver started at http://127.31.57.193:37783/ using document root <none> and password file <none>
I20260809 19:30:46.155102 11807 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:46.155164 11807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:46.155311 11807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:46.156253 11807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "30eebdcbd469457894d1fc8dd5e3af5c"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "10a1c13f4f480985b89cef63ce5882df"
server_key_iv: "d7cc7c7020e1928b941ff1abcd50cb04"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.156591 11807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "30eebdcbd469457894d1fc8dd5e3af5c"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "10a1c13f4f480985b89cef63ce5882df"
server_key_iv: "d7cc7c7020e1928b941ff1abcd50cb04"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.157794 11807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:46.158411 11824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.158562 11807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:46.158649 11807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "30eebdcbd469457894d1fc8dd5e3af5c"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "10a1c13f4f480985b89cef63ce5882df"
server_key_iv: "d7cc7c7020e1928b941ff1abcd50cb04"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.158726 11807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:46.166599 11807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:46.168187 11807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:46.168334 11807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:46.168756 11807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:46.168816 11807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.168846 11807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:46.168874 11807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.175323 11807 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:40531
I20260809 19:30:46.175428 12004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:40531 every 8 connection(s)
I20260809 19:30:46.175669 11807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:46.178149 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.169138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:46921 (local address 127.31.57.254:34769)
0809 19:30:46.169209 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.169211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.169359 (+ 148us) server_negotiation.cc:407] Connection header received
0809 19:30:46.169735 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.169736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.169750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.169832 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.170315 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.171113 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.172442 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.172749 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.172811 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.175497 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.175498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.175498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.175505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.176873 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.177259 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.177260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.177261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.177322 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.177557 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.177558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.177558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.177666 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.177727 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.177978 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.178065 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:30:46.178272 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.169121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.193:46921)
0809 19:30:46.169245 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.169277 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.169647 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.169875 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.169877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.170066 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.170216 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.170219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.171207 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.171208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.172342 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.172345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.172833 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.172834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.172975 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.173475 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.173485 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.175341 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.176965 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.176972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.176974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.177187 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.177393 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.177394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.177394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.177493 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.177731 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.177737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.177857 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.177945 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.178081 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":66,"threads_started":1}
I20260809 19:30:46.178699 12005 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34769
I20260809 19:30:46.178778 12005 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:46.178966 12005 heartbeater.cc:511] Master 127.31.57.254:34769 requested a full tablet report, sending...
I20260809 19:30:46.179466 11682 ts_manager.cc:194] Registered new tserver with Master: 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531)
I20260809 19:30:46.180184 11682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:46921
I20260809 19:30:46.182448 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 11807
I20260809 19:30:46.182552 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:46.182756 31975 external_mini_cluster.cc:1428] Setting key 3a8beb15656223af92b6c549e472a8f5
I20260809 19:30:46.185391 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.179737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.193:43717)
0809 19:30:46.179865 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.179873 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.179919 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.180458 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.180459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.180640 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.180709 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.180711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.181262 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.181263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.181713 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.181715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.182018 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.182019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.182080 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.182497 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.182513 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.182874 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.184429 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.184430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.184430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.184592 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.184781 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.184781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.184782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.184843 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.185120 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.185121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.185172 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.185247 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.185327 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260809 19:30:46.185487 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.179810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:43717 (local address 127.31.57.254:34769)
0809 19:30:46.180368 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.180369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.180377 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:46.180395 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.180396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.180404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.180451 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.180783 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.181183 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.181783 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.181951 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.182007 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.182959 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.182960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.182961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.182967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.184325 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.184660 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.184661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.184662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.184712 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.184911 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.184912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.184912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.185037 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.185110 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.185299 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.185377 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542}
WARNING: no policy specified for oryx/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
I20260809 19:30:46.207265 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34769
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:46.281314 12023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:46.281498 12023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:46.281533 12023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:46.282617 12023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:46.282683 12023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:46.282720 12023 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:46.282770 12023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:46.284293 12023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:46.284543 12023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:46.284749 12023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:46.289136 12030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:46.289114 12029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:46.289170 12023 server_base.cc:1034] running on GCE node
W20260809 19:30:46.289143 12034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:46.289542 12023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:46.289814 12023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:46.290971 12023 hybrid_clock.cc:648] HybridClock initialized: now 1786303846290956 us; error 26 us; skew 500 ppm
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:46.293282 12023 init.cc:377] Logged in from keytab as oryx/127.31.57.194@KRBTEST.COM (short username oryx)
I20260809 19:30:46.293709 12023 webserver.cc:460] Webserver started at http://127.31.57.194:33031/ using document root <none> and password file <none>
I20260809 19:30:46.293931 12023 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:46.293996 12023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:46.294130 12023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:46.295115 12023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "d64960e4466ffd386201302187140307"
server_key_iv: "ae9de75bc8ffd03fbf5e2c8ba3d6a161"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.295533 12023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "d64960e4466ffd386201302187140307"
server_key_iv: "ae9de75bc8ffd03fbf5e2c8ba3d6a161"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.296823 12023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:46.297473 12046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.297634 12023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:46.297727 12023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "d64960e4466ffd386201302187140307"
server_key_iv: "ae9de75bc8ffd03fbf5e2c8ba3d6a161"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.297811 12023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:46.314100 12023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:46.315901 12023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:46.316052 12023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:46.316505 12023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:46.316576 12023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.316605 12023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:46.316634 12023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.323926 12023 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:39269
I20260809 19:30:46.324025 12241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:39269 every 8 connection(s)
I20260809 19:30:46.324308 12023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:30:46.326094 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.316970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:45331 (local address 127.31.57.254:34769)
0809 19:30:46.317037 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.317039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.317260 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:30:46.317938 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.317940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.317951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.318005 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.318881 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.319711 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.320798 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.320995 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.321077 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.323473 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.323474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.323475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.323482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.325090 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.325426 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.325427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.325427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.325478 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.325640 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.325640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.325640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.325725 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.325769 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.325961 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.326019 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:30:46.326242 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.316943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.194:45331)
0809 19:30:46.317110 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.317155 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.317809 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.318077 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.318081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.318486 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.318782 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.318788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.319807 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.319808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.320694 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.320697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.321105 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.321107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.321256 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.321733 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.321742 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.323339 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.325158 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.325163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.325164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.325364 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.325517 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.325517 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.325517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.325602 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.325775 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.325780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.325881 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.325951 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.326070 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260809 19:30:46.326687 12242 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34769
I20260809 19:30:46.326771 12242 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:46.326957 12242 heartbeater.cc:511] Master 127.31.57.254:34769 requested a full tablet report, sending...
I20260809 19:30:46.327329 11685 ts_manager.cc:194] Registered new tserver with Master: 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269)
I20260809 19:30:46.327890 11685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:45331
I20260809 19:30:46.331512 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 12023
I20260809 19:30:46.331609 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:46.331823 31975 external_mini_cluster.cc:1428] Setting key fc634ace6c45d712482b1a0bad3e292d
I20260809 19:30:46.332818 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.327835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:41473 (local address 127.31.57.254:34769)
0809 19:30:46.327883 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.327885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.327968 (+ 83us) server_negotiation.cc:407] Connection header received
0809 19:30:46.328049 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.328049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.328060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.328107 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.328485 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.328911 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.329522 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.329684 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.329729 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.330508 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.330509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.330509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.330513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.331700 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.332048 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.332049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.332049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.332104 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.332282 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.332283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.332283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.332431 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.332503 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.332661 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.332730 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:30:46.332828 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.327773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.194:41473)
0809 19:30:46.327896 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.327906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.327981 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.328125 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.328126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.328337 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.328410 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.328412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.328998 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.328998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.329459 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.329460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.329740 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.329741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.329785 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.330178 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.330183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.330444 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.331774 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.331775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.331775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.331982 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.332161 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.332162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.332163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.332221 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.332512 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.332513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.332556 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.332634 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.332731 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
I20260809 19:30:46.355527 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34769
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:46.429524 12256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:46.429684 12256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:46.429704 12256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:46.430708 12256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:46.430744 12256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:46.430763 12256 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:30:46.430811 12256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:46.432139 12256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.57.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:46.432331 12256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:46.432551 12256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:46.437067 12263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:46.437096 12265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:46.437071 12268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:46.437279 12256 server_base.cc:1034] running on GCE node
I20260809 19:30:46.437534 12256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:46.437767 12256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:46.438935 12256 hybrid_clock.cc:648] HybridClock initialized: now 1786303846438925 us; error 28 us; skew 500 ppm
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:46.441207 12256 init.cc:377] Logged in from keytab as oryx/127.31.57.195@KRBTEST.COM (short username oryx)
I20260809 19:30:46.441532 12256 webserver.cc:460] Webserver started at http://127.31.57.195:40201/ using document root <none> and password file <none>
I20260809 19:30:46.441676 12256 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:46.441709 12256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:46.441787 12256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:46.442610 12256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "e9d284ccc6d14787b845f766d10a9c01"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "859893fbce574ab5e79d5f302cd76a90"
server_key_iv: "ebd49eaa8fbb85b5974df872e00e03aa"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.442883 12256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "e9d284ccc6d14787b845f766d10a9c01"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "859893fbce574ab5e79d5f302cd76a90"
server_key_iv: "ebd49eaa8fbb85b5974df872e00e03aa"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.444078 12256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:30:46.444734 12278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.444904 12256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:46.444990 12256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "e9d284ccc6d14787b845f766d10a9c01"
format_stamp: "Formatted at 2026-08-09 19:30:46 on dist-test-slave-x4qh"
server_key: "859893fbce574ab5e79d5f302cd76a90"
server_key_iv: "ebd49eaa8fbb85b5974df872e00e03aa"
server_key_version: "encryptionkey@0"
I20260809 19:30:46.445060 12256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:46.463466 12256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:46.465015 12256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:46.465161 12256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:46.465654 12256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:46.465706 12256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.465744 12256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:46.465793 12256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.471959 12256 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:34961
I20260809 19:30:46.472237 12256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:46.472587 12458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:34961 every 8 connection(s)
I20260809 19:30:46.474809 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.466023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:52913 (local address 127.31.57.254:34769)
0809 19:30:46.466063 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.466065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.466252 (+ 187us) server_negotiation.cc:407] Connection header received
0809 19:30:46.466653 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.466655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.466666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.466719 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.467423 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.468137 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.469609 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.469931 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.470012 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.472341 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.472343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.472343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.472350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.473520 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.473879 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.473880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.473881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.473936 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.474160 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.474161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.474162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.474300 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.474366 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.474623 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.474719 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:46.474931 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.466024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.195:52913)
0809 19:30:46.466143 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.466171 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.466562 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.466777 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.466780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.467054 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.467298 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.467304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.468254 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.468255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.469483 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.469487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.470047 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.470048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.470192 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.470699 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.470708 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.472253 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.473599 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.473605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.473607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.473808 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.473996 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.473997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.473997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.474094 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.474370 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.474376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.474498 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.474591 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.474742 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":69,"threads_started":1}
I20260809 19:30:46.475307 12460 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34769
I20260809 19:30:46.475423 12460 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:46.475589 12460 heartbeater.cc:511] Master 127.31.57.254:34769 requested a full tablet report, sending...
I20260809 19:30:46.476125 11682 ts_manager.cc:194] Registered new tserver with Master: e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:46.476593 11682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:52913
I20260809 19:30:46.479781 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 12256
I20260809 19:30:46.479889 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:46.480051 31975 external_mini_cluster.cc:1428] Setting key afb2b9d1e47d609fcdb7751a06fd40ba
I20260809 19:30:46.480934 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:30:46.481123 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.476207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:51795 (local address 127.31.57.254:34769)
0809 19:30:46.476235 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.476237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.476317 (+ 80us) server_negotiation.cc:407] Connection header received
0809 19:30:46.476371 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.476372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.476378 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.476416 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.476696 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.477193 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.477776 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.477934 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.477977 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.478649 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.478650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.478650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.478656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.480030 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.480364 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.480365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.480365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.480426 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.480617 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.480633 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.480633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.480754 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.480831 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.480991 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.481052 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 19:30:46.481139 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.476147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.31.57.195:51795)
0809 19:30:46.476264 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.476273 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.476324 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.476420 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.476421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.476575 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.476633 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.476634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.477297 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.477298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.477711 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.477713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.477988 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.477988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.478030 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.478332 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.478336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.478584 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.480099 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.480100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.480101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.480273 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.480494 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.480495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.480495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.480544 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.480830 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.480831 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.480882 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.480951 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.481045 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:30:46.500914 11773 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:46.496059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.31.57.254:34769)
0809 19:30:46.496094 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.496095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.496103 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:46.496141 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.496141 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.496147 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.496189 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.496686 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.497108 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.497756 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.497911 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.497953 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.500852 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60008: BlockingRecv error: recv got EOF from 127.0.0.1:60008 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.507267 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.501534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.31.57.254:34769)
0809 19:30:46.501565 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.501567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.501683 (+ 116us) server_negotiation.cc:407] Connection header received
0809 19:30:46.501743 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.501743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.501750 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.501804 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.502164 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.502544 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.503094 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.503264 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.503311 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.505169 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.505170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.505170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.505176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.506400 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.506662 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.506663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.506663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.506705 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.506875 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.506876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.506876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.507002 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.507080 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.507156 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.507205 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:30:46.508636 11682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:46.509977 11682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:46.515780 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.511989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.31.57.193:40531)
0809 19:30:46.512142 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.512146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.512155 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:46.512192 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.512193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.512204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.512290 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.512434 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.513230 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.514601 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.515305 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.515420 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.515495 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.515648 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.515689 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 19:30:46.515985 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.511813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.0.0.1:58232)
0809 19:30:46.511956 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.511991 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.512052 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.512303 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.512305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.512307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.512362 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.512364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.513308 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.513309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.514527 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.514530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.515443 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.515444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.515528 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.515542 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.515866 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.515926 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":50,"threads_started":1}
I20260809 19:30:46.516525 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.511845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34961 (local address 127.0.0.1:42450)
0809 19:30:46.512450 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.512459 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.512505 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.512664 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.512665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.512667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.512713 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.512714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.513505 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.513506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.514528 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.514530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.516296 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.516297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.516352 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.516356 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.516416 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.516453 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":5,"thread_start_us":35,"threads_started":1}
I20260809 19:30:46.516536 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.511988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.31.57.195:34961)
0809 19:30:46.512391 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.512395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.512501 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:30:46.512575 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.512577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.512590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.512655 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.512786 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.513417 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.515533 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.516196 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.516270 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.516351 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.516428 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.516461 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":80,"threads_started":1}
I20260809 19:30:46.516687 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.512159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.31.57.194:39269)
0809 19:30:46.512311 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.512314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.512366 (+ 52us) server_negotiation.cc:407] Connection header received
0809 19:30:46.513339 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.513341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.513353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.513432 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.513595 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.514196 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.515502 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.516203 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.516302 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.516372 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.516561 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.516603 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 19:30:46.516645 11897 tablet_service.cc:1467] Processing CreateTablet for tablet e2aa44e36498423a965c6ee227dcaade (DEFAULT_TABLE table=test-table [id=6a776757408549ff9c7fac1992dc661c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.516834 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.511813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39269 (local address 127.0.0.1:48436)
0809 19:30:46.512316 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.512325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.512379 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.513432 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.513433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.513435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.513524 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.513525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.514323 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.514324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.515438 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.515442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.516397 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.516398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.516446 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.516449 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.516491 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.516519 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":50,"thread_start_us":45,"threads_started":1}
I20260809 19:30:46.516957 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2aa44e36498423a965c6ee227dcaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.517474 12131 tablet_service.cc:1467] Processing CreateTablet for tablet e2aa44e36498423a965c6ee227dcaade (DEFAULT_TABLE table=test-table [id=6a776757408549ff9c7fac1992dc661c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.517757 12131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2aa44e36498423a965c6ee227dcaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.520845 12502 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Bootstrap starting.
I20260809 19:30:46.521450 12502 tablet_bootstrap.cc:654] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.521441 12503 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Bootstrap starting.
I20260809 19:30:46.521777 12502 log.cc:824] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:46.521883 12365 tablet_service.cc:1467] Processing CreateTablet for tablet e2aa44e36498423a965c6ee227dcaade (DEFAULT_TABLE table=test-table [id=6a776757408549ff9c7fac1992dc661c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.522100 12503 tablet_bootstrap.cc:654] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.522205 12365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2aa44e36498423a965c6ee227dcaade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.522425 12503 log.cc:824] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:46.522591 12502 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: No bootstrap required, opened a new log
I20260809 19:30:46.522661 12502 ts_tablet_manager.cc:1397] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:46.523085 12503 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: No bootstrap required, opened a new log
I20260809 19:30:46.523159 12503 ts_tablet_manager.cc:1397] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:46.524528 12503 raft_consensus.cc:348] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.524672 12503 raft_consensus.cc:374] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.524572 12502 raft_consensus.cc:348] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.524716 12503 raft_consensus.cc:729] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f1643b0c1c46d8875f0fd9ffbf2214, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.524744 12502 raft_consensus.cc:374] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.524793 12502 raft_consensus.cc:729] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.524850 12503 consensus_queue.cc:260] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.524922 12502 consensus_queue.cc:260] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.525130 12503 ts_tablet_manager.cc:1428] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:30:46.525197 12502 ts_tablet_manager.cc:1428] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:30:46.525255 12242 heartbeater.cc:503] Master 127.31.57.254:34769 was elected leader, sending a full tablet report...
I20260809 19:30:46.525337 12005 heartbeater.cc:503] Master 127.31.57.254:34769 was elected leader, sending a full tablet report...
I20260809 19:30:46.525638 12511 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Bootstrap starting.
I20260809 19:30:46.526228 12511 tablet_bootstrap.cc:654] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.526536 12511 log.cc:824] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:46.527083 12511 tablet_bootstrap.cc:492] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: No bootstrap required, opened a new log
I20260809 19:30:46.527148 12511 ts_tablet_manager.cc:1397] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:46.528527 12511 raft_consensus.cc:348] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.528630 12511 raft_consensus.cc:374] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.528647 12511 raft_consensus.cc:729] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.528733 12511 consensus_queue.cc:260] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.528923 12511 ts_tablet_manager.cc:1428] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:30:46.528980 12460 heartbeater.cc:503] Master 127.31.57.254:34769 was elected leader, sending a full tablet report...
W20260809 19:30:46.574849 12243 tablet.cc:2369] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:46.676327 12006 tablet.cc:2369] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:30:46.722810 12461 tablet.cc:2369] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:30:46.759172 12510 raft_consensus.cc:482] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:46.759277 12510 raft_consensus.cc:504] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.759688 12510 leader_election.cc:288] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:46.762524 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.759927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:33097 (local address 127.31.57.195:34961)
0809 19:30:46.759980 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.759981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.760078 (+ 97us) server_negotiation.cc:407] Connection header received
0809 19:30:46.760142 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.760143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.760152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.760212 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.760358 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.760818 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.761763 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.762195 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.762269 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.762278 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.762425 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.762459 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:30:46.762543 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.759859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34961 (local address 127.31.57.193:33097)
0809 19:30:46.760017 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.760024 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.760092 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.760215 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.760215 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.760217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.760289 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.760290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.760890 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.760891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.761675 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.761677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.762268 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.762269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.762346 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.762350 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.762417 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.762459 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":14,"thread_start_us":49,"threads_started":1}
I20260809 19:30:46.762542 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.759892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:44581 (local address 127.31.57.194:39269)
0809 19:30:46.759940 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.759942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.760041 (+ 99us) server_negotiation.cc:407] Connection header received
0809 19:30:46.760109 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.760110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.760124 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.760190 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.760336 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.760781 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.761764 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.762205 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.762285 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.762293 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.762433 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.762477 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:46.762562 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.759824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39269 (local address 127.31.57.193:44581)
0809 19:30:46.759973 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.759987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.760056 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.760196 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.760197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.760199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.760267 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.760269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.760876 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.760876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.761674 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.761677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.762293 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.762294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.762367 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.762370 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.762428 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.762477 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":41,"thread_start_us":76,"threads_started":1}
I20260809 19:30:46.762797 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2aa44e36498423a965c6ee227dcaade" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01" is_pre_election: true
I20260809 19:30:46.762905 12394 raft_consensus.cc:2454] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:46.763098 11831 leader_election.cc:302] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:46.763252 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2aa44e36498423a965c6ee227dcaade" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" is_pre_election: true
I20260809 19:30:46.763430 12170 raft_consensus.cc:2454] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:46.763530 12510 raft_consensus.cc:2781] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:46.763590 12510 raft_consensus.cc:482] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:46.763612 12510 raft_consensus.cc:3037] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.764369 12510 raft_consensus.cc:504] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.764504 12510 leader_election.cc:288] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Requested vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:46.764616 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2aa44e36498423a965c6ee227dcaade" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
I20260809 19:30:46.764647 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2aa44e36498423a965c6ee227dcaade" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01"
I20260809 19:30:46.764712 12394 raft_consensus.cc:3037] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.764705 12170 raft_consensus.cc:3037] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.765555 12394 raft_consensus.cc:2454] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:46.765576 12170 raft_consensus.cc:2454] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:46.765715 11833 leader_election.cc:302] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, 66f1643b0c1c46d8875f0fd9ffbf2214; no voters:
I20260809 19:30:46.765832 12510 raft_consensus.cc:2781] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:46.765965 12510 raft_consensus.cc:686] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Becoming Leader. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Running, Role: LEADER
I20260809 19:30:46.766052 12510 consensus_queue.cc:237] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.766728 11685 catalog_manager.cc:5495] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193). New cstate: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:46.802919 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.800893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.31.57.193:40531)
0809 19:30:46.800931 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.800933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.800941 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:46.800992 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.800992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.801019 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.801087 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:46.801222 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.801612 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.802370 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.802527 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.802571 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.802691 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:46.802792 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:46.802844 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.802860 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.802875 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:46.804551 12394 raft_consensus.cc:1261] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:46.804589 12170 raft_consensus.cc:1261] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:46.804883 12516 consensus_queue.cc:1035] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:46.804996 12510 consensus_queue.cc:1035] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:46.808266 12530 mvcc.cc:205] Tried to move back new op lower bound from 7316700556509044736 to 7316700556354314240. Current Snapshot: MvccSnapshot[applied={T|T < 7316700556509044736}]
I20260809 19:30:46.809672 11685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:46.809754 11685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:46.810410 11685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Sending DeleteTablet for 3 replicas of tablet e2aa44e36498423a965c6ee227dcaade
I20260809 19:30:46.810681 11897 tablet_service.cc:1514] Processing DeleteTablet for tablet e2aa44e36498423a965c6ee227dcaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:46 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:58232
I20260809 19:30:46.810731 12131 tablet_service.cc:1514] Processing DeleteTablet for tablet e2aa44e36498423a965c6ee227dcaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:46 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:48436
I20260809 19:30:46.810756 12365 tablet_service.cc:1514] Processing DeleteTablet for tablet e2aa44e36498423a965c6ee227dcaade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:46 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42450
I20260809 19:30:46.810945 12551 tablet_replica.cc:330] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: stopping tablet replica
I20260809 19:30:46.811048 12547 tablet_replica.cc:330] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: stopping tablet replica
I20260809 19:30:46.811082 12551 raft_consensus.cc:2229] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:46.811105 11685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:30:46.811221 12551 raft_consensus.cc:2258] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:30:46.811244 11685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:46.811658 12551 ts_tablet_manager.cc:1905] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:46.811452 12547 raft_consensus.cc:2229] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:46.811830 12547 raft_consensus.cc:2258] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:46.812572 12131 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4b90e567fd42258a5ad1b19264649b (DEFAULT_TABLE table=test-table [id=4275c4560cb449cdb4acafdd9e866e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.812685 12131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4b90e567fd42258a5ad1b19264649b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.813551 12503 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Bootstrap starting.
I20260809 19:30:46.813637 12551 ts_tablet_manager.cc:1918] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:46.813699 12551 log.cc:1196] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/e2aa44e36498423a965c6ee227dcaade
I20260809 19:30:46.813838 12365 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4b90e567fd42258a5ad1b19264649b (DEFAULT_TABLE table=test-table [id=4275c4560cb449cdb4acafdd9e866e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.813948 12365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4b90e567fd42258a5ad1b19264649b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.813958 12551 ts_tablet_manager.cc:1939] T e2aa44e36498423a965c6ee227dcaade P e9d284ccc6d14787b845f766d10a9c01: Deleting consensus metadata
I20260809 19:30:46.814047 12503 tablet_bootstrap.cc:654] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.814709 12503 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: No bootstrap required, opened a new log
I20260809 19:30:46.814772 12503 ts_tablet_manager.cc:1397] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:46.814931 12503 raft_consensus.cc:348] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.815001 12503 raft_consensus.cc:374] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.815025 12503 raft_consensus.cc:729] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f1643b0c1c46d8875f0fd9ffbf2214, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.815080 12503 consensus_queue.cc:260] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.815186 12503 ts_tablet_manager.cc:1428] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.815250 12511 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Bootstrap starting.
I20260809 19:30:46.815616 12511 tablet_bootstrap.cc:654] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.815912 11660 catalog_manager.cc:4841] TS e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961): tablet e2aa44e36498423a965c6ee227dcaade (table test-table [id=6a776757408549ff9c7fac1992dc661c]) successfully deleted
I20260809 19:30:46.816187 12511 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: No bootstrap required, opened a new log
I20260809 19:30:46.816236 12511 ts_tablet_manager.cc:1397] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:46.816385 12511 raft_consensus.cc:348] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.816450 12511 raft_consensus.cc:374] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.816473 12511 raft_consensus.cc:729] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.816519 12511 consensus_queue.cc:260] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.816599 12511 ts_tablet_manager.cc:1428] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.816788 12548 tablet_replica.cc:330] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: stopping tablet replica
I20260809 19:30:46.812511 11897 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4b90e567fd42258a5ad1b19264649b (DEFAULT_TABLE table=test-table [id=4275c4560cb449cdb4acafdd9e866e96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:46.816951 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4b90e567fd42258a5ad1b19264649b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.817858 12502 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Bootstrap starting.
I20260809 19:30:46.818261 12502 tablet_bootstrap.cc:654] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.818794 12548 raft_consensus.cc:2229] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:46.818904 12502 tablet_bootstrap.cc:492] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: No bootstrap required, opened a new log
I20260809 19:30:46.818949 12502 ts_tablet_manager.cc:1397] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:46.819064 12502 raft_consensus.cc:348] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.819095 12547 ts_tablet_manager.cc:1905] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:46.819115 12502 raft_consensus.cc:374] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.819135 12502 raft_consensus.cc:729] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.819177 12502 consensus_queue.cc:260] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.819248 12502 ts_tablet_manager.cc:1428] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.820466 12548 raft_consensus.cc:2258] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:46.820462 12547 ts_tablet_manager.cc:1918] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:46.820535 12547 log.cc:1196] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/e2aa44e36498423a965c6ee227dcaade
I20260809 19:30:46.820958 12547 ts_tablet_manager.cc:1939] T e2aa44e36498423a965c6ee227dcaade P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting consensus metadata
I20260809 19:30:46.821102 12548 ts_tablet_manager.cc:1905] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:46.821378 11661 catalog_manager.cc:4841] TS 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531): tablet e2aa44e36498423a965c6ee227dcaade (table test-table [id=6a776757408549ff9c7fac1992dc661c]) successfully deleted
I20260809 19:30:46.822216 12548 ts_tablet_manager.cc:1918] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:46.822273 12548 log.cc:1196] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/e2aa44e36498423a965c6ee227dcaade
I20260809 19:30:46.822448 12548 ts_tablet_manager.cc:1939] T e2aa44e36498423a965c6ee227dcaade P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting consensus metadata
I20260809 19:30:46.822769 11662 catalog_manager.cc:4841] TS 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269): tablet e2aa44e36498423a965c6ee227dcaade (table test-table [id=6a776757408549ff9c7fac1992dc661c]) successfully deleted
I20260809 19:30:46.828758 12513 raft_consensus.cc:482] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:46.828830 12513 raft_consensus.cc:504] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.829074 12513 leader_election.cc:288] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531), 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269)
I20260809 19:30:46.832096 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.829384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:45173 (local address 127.31.57.194:39269)
0809 19:30:46.829431 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.829433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.829457 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:30:46.829476 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.829477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.829485 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.829530 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.829757 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.830386 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831450 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.831759 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831842 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.831851 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.831990 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.832024 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:30:46.832100 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.829214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39269 (local address 127.31.57.195:45173)
0809 19:30:46.829369 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.829379 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.829426 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.829620 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.829621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.829622 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.829672 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.829673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.830460 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.830460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.831343 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.831344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.831843 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.831914 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.831916 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.831983 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.832023 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":10,"thread_start_us":39,"threads_started":1}
I20260809 19:30:46.832159 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.829251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50707 (local address 127.31.57.193:40531)
0809 19:30:46.829295 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.829297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.829427 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:30:46.829529 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.829529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.829538 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.829582 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:30:46.829728 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.830132 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831294 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.831805 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831865 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.831872 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.832072 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.832104 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:46.832326 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4b90e567fd42258a5ad1b19264649b" candidate_uuid: "e9d284ccc6d14787b845f766d10a9c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" is_pre_election: true
I20260809 19:30:46.832409 12170 raft_consensus.cc:2454] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d284ccc6d14787b845f766d10a9c01 in term 0.
I20260809 19:30:46.832535 12282 leader_election.cc:302] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f1643b0c1c46d8875f0fd9ffbf2214, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:46.832681 12513 raft_consensus.cc:2781] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:46.832698 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.829207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.31.57.195:50707)
0809 19:30:46.829359 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.829370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.829450 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.829583 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.829584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.829585 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:30:46.829651 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.829652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.830205 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.830206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.831214 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.831216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.831921 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.831922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.831986 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.831989 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.832593 (+ 604us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.832633 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 19:30:46.832742 12513 raft_consensus.cc:482] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:46.832768 12513 raft_consensus.cc:3037] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.832970 11925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4b90e567fd42258a5ad1b19264649b" candidate_uuid: "e9d284ccc6d14787b845f766d10a9c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" is_pre_election: true
I20260809 19:30:46.833088 11925 raft_consensus.cc:2454] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d284ccc6d14787b845f766d10a9c01 in term 0.
I20260809 19:30:46.833453 12513 raft_consensus.cc:504] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.833568 12513 leader_election.cc:288] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [CANDIDATE]: Term 1 election: Requested vote from peers 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531), 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269)
I20260809 19:30:46.833683 11925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4b90e567fd42258a5ad1b19264649b" candidate_uuid: "e9d284ccc6d14787b845f766d10a9c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebdcbd469457894d1fc8dd5e3af5c"
I20260809 19:30:46.833740 11925 raft_consensus.cc:3037] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.833717 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4b90e567fd42258a5ad1b19264649b" candidate_uuid: "e9d284ccc6d14787b845f766d10a9c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
I20260809 19:30:46.833773 12170 raft_consensus.cc:3037] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:46.834304 12170 raft_consensus.cc:2454] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d284ccc6d14787b845f766d10a9c01 in term 1.
I20260809 19:30:46.834444 12282 leader_election.cc:302] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66f1643b0c1c46d8875f0fd9ffbf2214, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:46.834481 11925 raft_consensus.cc:2454] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d284ccc6d14787b845f766d10a9c01 in term 1.
I20260809 19:30:46.834510 12513 raft_consensus.cc:2781] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:46.834623 12513 raft_consensus.cc:686] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 1 LEADER]: Becoming Leader. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Running, Role: LEADER
I20260809 19:30:46.834709 12513 consensus_queue.cc:237] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.835249 11685 catalog_manager.cc:5495] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195). New cstate: current_term: 1 leader_uuid: "e9d284ccc6d14787b845f766d10a9c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } health_report { overall_health: HEALTHY } } }
Aug 09 19:30:46 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303845, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.254@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:46.858913 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.851963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34769 (local address 127.0.0.1:60032)
0809 19:30:46.852105 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:46.852114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:46.852174 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:46.852304 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:46.852305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:46.852641 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:46.852689 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.852692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.853238 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.853239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.854025 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:46.854027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.854357 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.854358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:46.854432 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.854943 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:46.854947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:46.856401 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:46.858019 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.858026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.858027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.858209 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.858398 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:46.858399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:46.858400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:46.858451 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:46.858680 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:46.858685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:46.858722 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 19:30:46.858768 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:30:46.858850 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260809 19:30:46.859010 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:46.852007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.31.57.254:34769)
0809 19:30:46.852074 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:46.852075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:46.852166 (+ 91us) server_negotiation.cc:407] Connection header received
0809 19:30:46.852226 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:46.852227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:46.852234 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:46.852283 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:46.852756 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.853173 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.854103 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:46.854283 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:46.854327 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:46.856484 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:46.856485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:46.856486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:46.856494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:46.857936 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.858270 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.858271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.858272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.858336 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:46.858506 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:46.858507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:46.858507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:46.858625 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:46.858688 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:46.858807 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 19:30:46.858909 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":44}
I20260809 19:30:46.861171 11685 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:60032:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:30:46.861404 11685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:46.862658 11897 tablet_service.cc:1467] Processing CreateTablet for tablet a5f636add08b40f7ab7db60b874f4322 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5e7124647a7400896fb6792ae2b625d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:46.862823 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f636add08b40f7ab7db60b874f4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.862782 12131 tablet_service.cc:1467] Processing CreateTablet for tablet a5f636add08b40f7ab7db60b874f4322 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5e7124647a7400896fb6792ae2b625d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:46.862936 12131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f636add08b40f7ab7db60b874f4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.864042 12502 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: Bootstrap starting.
I20260809 19:30:46.864359 12503 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: Bootstrap starting.
I20260809 19:30:46.864459 12502 tablet_bootstrap.cc:654] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.864434 12365 tablet_service.cc:1467] Processing CreateTablet for tablet a5f636add08b40f7ab7db60b874f4322 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5e7124647a7400896fb6792ae2b625d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:30:46.864559 12365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f636add08b40f7ab7db60b874f4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:46.864859 12503 tablet_bootstrap.cc:654] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.865494 12503 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: No bootstrap required, opened a new log
I20260809 19:30:46.865553 12503 ts_tablet_manager.cc:1397] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:46.865705 12503 raft_consensus.cc:348] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.865770 12503 raft_consensus.cc:374] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.865803 12503 raft_consensus.cc:729] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f1643b0c1c46d8875f0fd9ffbf2214, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.865835 12503 consensus_queue.cc:260] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.865919 12503 ts_tablet_manager.cc:1428] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:46.865919 12511 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: Bootstrap starting.
I20260809 19:30:46.866035 12508 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:46.866109 12508 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:46.866319 12511 tablet_bootstrap.cc:654] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:46.866444 12502 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: No bootstrap required, opened a new log
I20260809 19:30:46.866484 12502 ts_tablet_manager.cc:1397] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:46.866597 12502 raft_consensus.cc:348] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.866652 12502 raft_consensus.cc:374] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.866670 12502 raft_consensus.cc:729] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.866706 12502 consensus_queue.cc:260] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.866777 12502 ts_tablet_manager.cc:1428] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:46.866878 12537 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:46.866945 12537 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:46.866981 12511 tablet_bootstrap.cc:492] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: No bootstrap required, opened a new log
I20260809 19:30:46.867031 12511 ts_tablet_manager.cc:1397] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:30:46.867177 12511 raft_consensus.cc:348] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.867245 12511 raft_consensus.cc:374] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:46.867270 12511 raft_consensus.cc:729] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Initialized, Role: FOLLOWER
I20260809 19:30:46.867324 12511 consensus_queue.cc:260] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:46.867439 12511 ts_tablet_manager.cc:1428] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:46.867468 12513 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:46.867542 12513 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:47.086558 12537 raft_consensus.cc:482] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:47.086678 12537 raft_consensus.cc:504] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.086872 12537 leader_election.cc:288] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.087061 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f636add08b40f7ab7db60b874f4322" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01" is_pre_election: true
I20260809 19:30:47.087075 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f636add08b40f7ab7db60b874f4322" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" is_pre_election: true
I20260809 19:30:47.087158 12394 raft_consensus.cc:2454] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.087162 12170 raft_consensus.cc:2454] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.087306 11831 leader_election.cc:302] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:47.087414 12537 raft_consensus.cc:2781] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:47.087451 12537 raft_consensus.cc:482] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:47.087467 12537 raft_consensus.cc:3037] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.088176 12537 raft_consensus.cc:504] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.088339 12537 leader_election.cc:288] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Requested vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.088491 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f636add08b40f7ab7db60b874f4322" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01"
I20260809 19:30:47.088497 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f636add08b40f7ab7db60b874f4322" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
I20260809 19:30:47.088570 12394 raft_consensus.cc:3037] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.088572 12170 raft_consensus.cc:3037] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.089193 12394 raft_consensus.cc:2454] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.089193 12170 raft_consensus.cc:2454] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.089398 11831 leader_election.cc:302] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:47.089514 12537 raft_consensus.cc:2781] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:47.089572 12537 raft_consensus.cc:686] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Becoming Leader. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Running, Role: LEADER
I20260809 19:30:47.089625 12537 consensus_queue.cc:237] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.089897 12516 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: TxnStatusTablet state changed. Reason: New leader 30eebdcbd469457894d1fc8dd5e3af5c. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.090005 12516 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:47.090226 12576 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:30:47.090212 11685 catalog_manager.cc:5495] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193). New cstate: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:47.090339 12576 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:30:47.150046 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.147535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.0.0.1:58248)
0809 19:30:47.147623 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:47.147633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:47.147697 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:47.147851 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:47.147851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:47.147852 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:47.147899 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.147900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.148732 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.148732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.149244 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.149245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.149675 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.149675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.149721 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.149735 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.149923 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.149929 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:30:47.149957 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:30:47.149995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 19:30:47.150058 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.147581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.31.57.193:40531)
0809 19:30:47.147629 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.147631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.147684 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:30:47.147749 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.147750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.147760 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.147825 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.147970 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.148659 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.149335 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.149599 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.149676 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.149818 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.149853 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.149919 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.149974 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.149992 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:30:47.150882 12394 raft_consensus.cc:1261] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.150892 12170 raft_consensus.cc:1261] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.151074 12516 consensus_queue.cc:1035] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.151154 12537 consensus_queue.cc:1035] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.151664 12513 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: TxnStatusTablet state changed. Reason: New leader 30eebdcbd469457894d1fc8dd5e3af5c. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.151749 12513 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:47.152114 12508 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: TxnStatusTablet state changed. Reason: New leader 30eebdcbd469457894d1fc8dd5e3af5c. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.152194 12508 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:47.152390 12516 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.152465 12516 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:47.152514 12527 mvcc.cc:205] Tried to move back new op lower bound from 7316700557928599552 to 7316700557679431680. Current Snapshot: MvccSnapshot[applied={T|T < 7316700557928599552 or (T in {7316700557928599552})}]
I20260809 19:30:47.152945 12535 mvcc.cc:205] Tried to move back new op lower bound from 7316700557928599552 to 7316700557679431680. Current Snapshot: MvccSnapshot[applied={T|T < 7316700557928599552 or (T in {7316700557928599552})}]
I20260809 19:30:47.153033 12513 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.153103 12513 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P e9d284ccc6d14787b845f766d10a9c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:47.153556 12516 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.153620 12516 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 30eebdcbd469457894d1fc8dd5e3af5c: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:30:47.153968 12508 tablet_replica.cc:439] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } } }
I20260809 19:30:47.154032 12508 tablet_replica.cc:442] T a5f636add08b40f7ab7db60b874f4322 P 66f1643b0c1c46d8875f0fd9ffbf2214: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:30:47.156581 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.154023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.31.57.195:34961)
0809 19:30:47.154088 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.154090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.154098 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:47.154129 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.154130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.154148 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.154215 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.154371 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.154976 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.155738 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.155962 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.156029 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.156197 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.156325 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.156421 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.156472 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.156490 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:30:47.161099 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.158699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:53579 (local address 127.31.57.193:40531)
0809 19:30:47.158735 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.158736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.158866 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:30:47.158937 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.158937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.158944 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.158983 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.159106 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.159545 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.160539 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.160702 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.160745 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.160947 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.160964 (+ 17us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.160989 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.161051 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.161063 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:30:47.161283 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.158610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.31.57.195:53579)
0809 19:30:47.158799 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:47.158809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:47.158868 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:47.158990 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:47.158990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:47.158991 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:47.159051 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.159052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.159625 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.159627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.160460 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.160463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.160770 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.160771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.160860 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.160875 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.161004 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.161006 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:47.161037 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:30:47.161075 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260809 19:30:47.163513 12170 raft_consensus.cc:1261] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Refusing update from remote peer e9d284ccc6d14787b845f766d10a9c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.163561 11925 raft_consensus.cc:1261] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Refusing update from remote peer e9d284ccc6d14787b845f766d10a9c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.163793 12513 consensus_queue.cc:1035] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.164108 12556 consensus_queue.cc:1035] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.171984 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.169726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59915 (local address 127.31.57.195:34961)
0809 19:30:47.169763 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.169764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.169774 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:47.169859 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.169860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.169871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.169936 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.170046 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.170636 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.171255 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.171526 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.171585 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.171779 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.171816 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.171861 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.171932 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.171947 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:47.172255 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.169592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34961 (local address 127.31.57.193:59915)
0809 19:30:47.169726 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:47.169736 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:47.169787 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:47.169952 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:47.169953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:47.169954 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:47.169989 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.169990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.170700 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.170700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.171183 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.171184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.171619 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.171621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.171672 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.171687 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.171880 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.171882 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:30:47.171931 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:30:47.171973 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260809 19:30:47.183072 11685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.183189 11685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.183864 11685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Sending DeleteTablet for 3 replicas of tablet 0f4b90e567fd42258a5ad1b19264649b
I20260809 19:30:47.184171 11897 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4b90e567fd42258a5ad1b19264649b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:58232
I20260809 19:30:47.184278 12131 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4b90e567fd42258a5ad1b19264649b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:48436
I20260809 19:30:47.184290 12547 tablet_replica.cc:330] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: stopping tablet replica
I20260809 19:30:47.184365 12547 raft_consensus.cc:2229] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.184384 12548 tablet_replica.cc:330] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: stopping tablet replica
I20260809 19:30:47.184423 12547 raft_consensus.cc:2258] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.184442 12548 raft_consensus.cc:2229] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.184433 12365 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4b90e567fd42258a5ad1b19264649b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42450
I20260809 19:30:47.184476 12548 raft_consensus.cc:2258] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.184582 12551 tablet_replica.cc:330] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: stopping tablet replica
I20260809 19:30:47.184650 12551 raft_consensus.cc:2229] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:47.184731 12551 raft_consensus.cc:2258] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.185290 12551 ts_tablet_manager.cc:1905] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.187256 12551 ts_tablet_manager.cc:1918] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:47.187312 12551 log.cc:1196] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/0f4b90e567fd42258a5ad1b19264649b
I20260809 19:30:47.187566 12551 ts_tablet_manager.cc:1939] T 0f4b90e567fd42258a5ad1b19264649b P e9d284ccc6d14787b845f766d10a9c01: Deleting consensus metadata
I20260809 19:30:47.187978 12547 ts_tablet_manager.cc:1905] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.189638 12547 ts_tablet_manager.cc:1918] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:47.189694 12547 log.cc:1196] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/0f4b90e567fd42258a5ad1b19264649b
I20260809 19:30:47.189890 12547 ts_tablet_manager.cc:1939] T 0f4b90e567fd42258a5ad1b19264649b P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting consensus metadata
I20260809 19:30:47.190339 11661 catalog_manager.cc:4841] TS 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531): tablet 0f4b90e567fd42258a5ad1b19264649b (table test-table [id=4275c4560cb449cdb4acafdd9e866e96]) successfully deleted
I20260809 19:30:47.190527 11660 catalog_manager.cc:4841] TS e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961): tablet 0f4b90e567fd42258a5ad1b19264649b (table test-table [id=4275c4560cb449cdb4acafdd9e866e96]) successfully deleted
I20260809 19:30:47.190790 12548 ts_tablet_manager.cc:1905] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.192575 12548 ts_tablet_manager.cc:1918] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:47.192634 12548 log.cc:1196] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/0f4b90e567fd42258a5ad1b19264649b
I20260809 19:30:47.192860 12548 ts_tablet_manager.cc:1939] T 0f4b90e567fd42258a5ad1b19264649b P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting consensus metadata
I20260809 19:30:47.193166 11662 catalog_manager.cc:4841] TS 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269): tablet 0f4b90e567fd42258a5ad1b19264649b (table test-table [id=4275c4560cb449cdb4acafdd9e866e96]) successfully deleted
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[11615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:30:47.208652 11773 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:47.203862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.31.57.254:34769)
0809 19:30:47.203902 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.203904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.204025 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:30:47.204096 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.204097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.204106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.204163 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:47.204749 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.205223 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.205896 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.206066 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.206130 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.208591 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60044: BlockingRecv error: recv got EOF from 127.0.0.1:60044 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[11615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM
I20260809 19:30:47.216061 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.209491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.31.57.254:34769)
0809 19:30:47.209524 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.209526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.209746 (+ 220us) server_negotiation.cc:407] Connection header received
0809 19:30:47.209820 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.209821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.209829 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.209876 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:47.210377 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.210812 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.211494 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.211674 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.211749 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.213680 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:47.213681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:47.213681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:47.213686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:47.214996 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:47.215412 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:47.215413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:47.215413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:47.215478 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:47.215686 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:47.215687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:47.215687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:47.215776 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:47.215845 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.215963 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.216014 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:47.217376 11685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:47.217512 11685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:47.218735 11897 tablet_service.cc:1467] Processing CreateTablet for tablet bc33646219874680ba2c40d4c85c6aa1 (DEFAULT_TABLE table=test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.218792 12131 tablet_service.cc:1467] Processing CreateTablet for tablet bc33646219874680ba2c40d4c85c6aa1 (DEFAULT_TABLE table=test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.218847 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc33646219874680ba2c40d4c85c6aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.218905 12131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc33646219874680ba2c40d4c85c6aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.219866 12502 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Bootstrap starting.
I20260809 19:30:47.220072 12503 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Bootstrap starting.
I20260809 19:30:47.220168 12502 tablet_bootstrap.cc:654] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.220393 12365 tablet_service.cc:1467] Processing CreateTablet for tablet bc33646219874680ba2c40d4c85c6aa1 (DEFAULT_TABLE table=test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.220431 12503 tablet_bootstrap.cc:654] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.220495 12365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc33646219874680ba2c40d4c85c6aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.220988 12503 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: No bootstrap required, opened a new log
I20260809 19:30:47.221024 12503 ts_tablet_manager.cc:1397] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:47.221155 12503 raft_consensus.cc:348] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.221206 12503 raft_consensus.cc:374] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.221223 12503 raft_consensus.cc:729] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f1643b0c1c46d8875f0fd9ffbf2214, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.221274 12503 consensus_queue.cc:260] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.221345 12503 ts_tablet_manager.cc:1428] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:47.221457 12511 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Bootstrap starting.
I20260809 19:30:47.221908 12511 tablet_bootstrap.cc:654] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.221944 12502 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: No bootstrap required, opened a new log
I20260809 19:30:47.221989 12502 ts_tablet_manager.cc:1397] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:30:47.222069 12502 raft_consensus.cc:348] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.222121 12502 raft_consensus.cc:374] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.222136 12502 raft_consensus.cc:729] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.222200 12502 consensus_queue.cc:260] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.222293 12502 ts_tablet_manager.cc:1428] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:47.222532 12511 tablet_bootstrap.cc:492] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: No bootstrap required, opened a new log
I20260809 19:30:47.222581 12511 ts_tablet_manager.cc:1397] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:47.222705 12511 raft_consensus.cc:348] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.222767 12511 raft_consensus.cc:374] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.222785 12511 raft_consensus.cc:729] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.222820 12511 consensus_queue.cc:260] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.222900 12511 ts_tablet_manager.cc:1428] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:47.234432 12537 raft_consensus.cc:482] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:47.234493 12537 raft_consensus.cc:504] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.234614 12537 leader_election.cc:288] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.234766 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc33646219874680ba2c40d4c85c6aa1" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" is_pre_election: true
I20260809 19:30:47.234838 12170 raft_consensus.cc:2454] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.234941 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc33646219874680ba2c40d4c85c6aa1" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01" is_pre_election: true
I20260809 19:30:47.234961 11833 leader_election.cc:302] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, 66f1643b0c1c46d8875f0fd9ffbf2214; no voters:
I20260809 19:30:47.235008 12394 raft_consensus.cc:2454] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.235044 12537 raft_consensus.cc:2781] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:47.235075 12537 raft_consensus.cc:482] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:47.235098 12537 raft_consensus.cc:3037] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.235734 12537 raft_consensus.cc:504] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.235865 12537 leader_election.cc:288] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Requested vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.235996 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc33646219874680ba2c40d4c85c6aa1" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
I20260809 19:30:47.236061 12170 raft_consensus.cc:3037] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.236059 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc33646219874680ba2c40d4c85c6aa1" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01"
I20260809 19:30:47.236114 12394 raft_consensus.cc:3037] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.236766 12170 raft_consensus.cc:2454] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.236930 11833 leader_election.cc:302] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, 66f1643b0c1c46d8875f0fd9ffbf2214; no voters:
I20260809 19:30:47.237049 12394 raft_consensus.cc:2454] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.237159 12537 raft_consensus.cc:2781] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:47.237206 12537 raft_consensus.cc:686] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Becoming Leader. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Running, Role: LEADER
I20260809 19:30:47.237241 12537 consensus_queue.cc:237] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.237673 11682 catalog_manager.cc:5495] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193). New cstate: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:47.246228 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.244112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58258 (local address 127.31.57.193:40531)
0809 19:30:47.244150 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.244151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.244159 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:47.244209 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.244209 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.244218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.244280 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.244412 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.244909 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.245620 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.245806 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.245869 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.246007 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.246040 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.246083 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.246146 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.246162 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:47.246836 12394 raft_consensus.cc:1261] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.246876 12170 raft_consensus.cc:1261] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.247047 12516 consensus_queue.cc:1035] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.247134 12537 consensus_queue.cc:1035] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.249346 11685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.249414 11685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.249864 11685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Sending DeleteTablet for 3 replicas of tablet bc33646219874680ba2c40d4c85c6aa1
I20260809 19:30:47.250047 12131 tablet_service.cc:1514] Processing DeleteTablet for tablet bc33646219874680ba2c40d4c85c6aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:48436
I20260809 19:30:47.250129 11897 tablet_service.cc:1514] Processing DeleteTablet for tablet bc33646219874680ba2c40d4c85c6aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:58232
I20260809 19:30:47.250206 12130 tablet_service.cc:1514] Processing DeleteTablet for tablet bc33646219874680ba2c40d4c85c6aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:48436
I20260809 19:30:47.250211 12365 tablet_service.cc:1514] Processing DeleteTablet for tablet bc33646219874680ba2c40d4c85c6aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42450
I20260809 19:30:47.250276 12548 tablet_replica.cc:330] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: stopping tablet replica
I20260809 19:30:47.250313 12548 raft_consensus.cc:2229] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.250311 12551 tablet_replica.cc:330] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: stopping tablet replica
I20260809 19:30:47.250341 12548 raft_consensus.cc:2258] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.250355 12551 raft_consensus.cc:2229] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.250383 12551 raft_consensus.cc:2258] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.250396 12547 tablet_replica.cc:330] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: stopping tablet replica
I20260809 19:30:47.250440 12547 raft_consensus.cc:2229] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:47.250483 12547 raft_consensus.cc:2258] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.250607 12551 ts_tablet_manager.cc:1905] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.250882 12548 ts_tablet_manager.cc:1905] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.251614 12547 ts_tablet_manager.cc:1905] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.251586 11685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:30:47.251713 11685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:30:47.252254 12548 ts_tablet_manager.cc:1918] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:47.252306 12548 log.cc:1196] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/bc33646219874680ba2c40d4c85c6aa1
I20260809 19:30:47.252424 12551 ts_tablet_manager.cc:1918] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:47.252468 12551 log.cc:1196] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/bc33646219874680ba2c40d4c85c6aa1
I20260809 19:30:47.252511 12548 ts_tablet_manager.cc:1939] T bc33646219874680ba2c40d4c85c6aa1 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting consensus metadata
I20260809 19:30:47.252650 12551 ts_tablet_manager.cc:1939] T bc33646219874680ba2c40d4c85c6aa1 P e9d284ccc6d14787b845f766d10a9c01: Deleting consensus metadata
I20260809 19:30:47.252897 11660 catalog_manager.cc:4841] TS e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961): tablet bc33646219874680ba2c40d4c85c6aa1 (table test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]) successfully deleted
I20260809 19:30:47.252903 11662 catalog_manager.cc:4841] TS 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269): tablet bc33646219874680ba2c40d4c85c6aa1 (table test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]) successfully deleted
I20260809 19:30:47.252991 12547 ts_tablet_manager.cc:1918] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:30:47.253039 12547 log.cc:1196] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/bc33646219874680ba2c40d4c85c6aa1
W20260809 19:30:47.253038 11662 catalog_manager.cc:4808] TS 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269): delete failed for tablet bc33646219874680ba2c40d4c85c6aa1 because the tablet was not found. No further retry: Not found: Tablet not found: bc33646219874680ba2c40d4c85c6aa1
I20260809 19:30:47.253260 12547 ts_tablet_manager.cc:1939] T bc33646219874680ba2c40d4c85c6aa1 P 30eebdcbd469457894d1fc8dd5e3af5c: Deleting consensus metadata
I20260809 19:30:47.253536 11661 catalog_manager.cc:4841] TS 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531): tablet bc33646219874680ba2c40d4c85c6aa1 (table test-table [id=cbc94d8a551045ebbbebfa6c45c23a6d]) successfully deleted
I20260809 19:30:47.254356 11897 tablet_service.cc:1467] Processing CreateTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 (DEFAULT_TABLE table=test-table [id=26de0cff89c84094a1147b40edf2851b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.254359 12130 tablet_service.cc:1467] Processing CreateTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 (DEFAULT_TABLE table=test-table [id=26de0cff89c84094a1147b40edf2851b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.254359 12365 tablet_service.cc:1467] Processing CreateTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 (DEFAULT_TABLE table=test-table [id=26de0cff89c84094a1147b40edf2851b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:30:47.254460 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6a6f146c0c4a189580938b3ca38cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.254462 12365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6a6f146c0c4a189580938b3ca38cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.254460 12130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6a6f146c0c4a189580938b3ca38cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:47.255637 12511 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Bootstrap starting.
I20260809 19:30:47.255888 12511 tablet_bootstrap.cc:654] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.255991 12503 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Bootstrap starting.
I20260809 19:30:47.256139 12502 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: Bootstrap starting.
I20260809 19:30:47.256356 12503 tablet_bootstrap.cc:654] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.256408 12502 tablet_bootstrap.cc:654] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:47.256928 12502 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: No bootstrap required, opened a new log
I20260809 19:30:47.256979 12502 ts_tablet_manager.cc:1397] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:30:47.257131 12503 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: No bootstrap required, opened a new log
I20260809 19:30:47.257105 12502 raft_consensus.cc:348] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.257169 12502 raft_consensus.cc:374] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.257180 12503 ts_tablet_manager.cc:1397] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:30:47.257193 12502 raft_consensus.cc:729] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.257239 12502 consensus_queue.cc:260] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.257347 12502 ts_tablet_manager.cc:1428] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:47.257324 12503 raft_consensus.cc:348] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.257401 12503 raft_consensus.cc:374] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.257434 12503 raft_consensus.cc:729] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f1643b0c1c46d8875f0fd9ffbf2214, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.257472 12503 consensus_queue.cc:260] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.257566 12503 ts_tablet_manager.cc:1428] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:47.257943 12511 tablet_bootstrap.cc:492] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: No bootstrap required, opened a new log
I20260809 19:30:47.257993 12511 ts_tablet_manager.cc:1397] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 19:30:47.258178 12511 raft_consensus.cc:348] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.258260 12511 raft_consensus.cc:374] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:47.258301 12511 raft_consensus.cc:729] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d284ccc6d14787b845f766d10a9c01, State: Initialized, Role: FOLLOWER
I20260809 19:30:47.258349 12511 consensus_queue.cc:260] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.258447 12511 ts_tablet_manager.cc:1428] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:47.289883 12516 raft_consensus.cc:482] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:30:47.289958 12516 raft_consensus.cc:504] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.290105 12516 leader_election.cc:288] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.290279 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6a6f146c0c4a189580938b3ca38cd5" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01" is_pre_election: true
I20260809 19:30:47.290305 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6a6f146c0c4a189580938b3ca38cd5" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" is_pre_election: true
I20260809 19:30:47.290364 12394 raft_consensus.cc:2454] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.290373 12170 raft_consensus.cc:2454] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 0.
I20260809 19:30:47.290496 11833 leader_election.cc:302] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, 66f1643b0c1c46d8875f0fd9ffbf2214; no voters:
I20260809 19:30:47.290596 12516 raft_consensus.cc:2781] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:30:47.290635 12516 raft_consensus.cc:482] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:30:47.290654 12516 raft_consensus.cc:3037] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.291574 12516 raft_consensus.cc:504] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.291700 12516 leader_election.cc:288] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Requested vote from peers 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269), e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961)
I20260809 19:30:47.291831 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6a6f146c0c4a189580938b3ca38cd5" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214"
I20260809 19:30:47.291868 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6a6f146c0c4a189580938b3ca38cd5" candidate_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d284ccc6d14787b845f766d10a9c01"
I20260809 19:30:47.291918 12170 raft_consensus.cc:3037] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.291936 12394 raft_consensus.cc:3037] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:47.292433 12394 raft_consensus.cc:2454] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.292598 11831 leader_election.cc:302] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebdcbd469457894d1fc8dd5e3af5c, e9d284ccc6d14787b845f766d10a9c01; no voters:
I20260809 19:30:47.292683 12516 raft_consensus.cc:2781] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:47.292701 12170 raft_consensus.cc:2454] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebdcbd469457894d1fc8dd5e3af5c in term 1.
I20260809 19:30:47.292728 12516 raft_consensus.cc:686] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Becoming Leader. State: Replica: 30eebdcbd469457894d1fc8dd5e3af5c, State: Running, Role: LEADER
I20260809 19:30:47.292778 12516 consensus_queue.cc:237] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } }
I20260809 19:30:47.293362 11682 catalog_manager.cc:5495] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193). New cstate: current_term: 1 leader_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebdcbd469457894d1fc8dd5e3af5c" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 } health_report { overall_health: UNKNOWN } } }
I20260809 19:30:47.308178 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.306073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:41799 (local address 127.31.57.193:40531)
0809 19:30:47.306107 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:47.306109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:47.306117 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:47.306180 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:47.306181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:47.306190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:47.306240 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:47.306354 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.306806 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.307553 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.307730 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.307801 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.307937 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.307967 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.308013 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:47.308072 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:30:47.308108 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":17}
I20260809 19:30:47.308178 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.306016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.31.57.193:41799)
0809 19:30:47.306067 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:47.306078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:47.306123 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:47.306242 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:47.306243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:47.306244 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:30:47.306298 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.306299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.306880 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.306881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.307460 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:47.307463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:47.307801 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:47.307802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:47.307851 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:47.307857 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:30:47.308018 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:30:47.308019 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:30:47.308069 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:30:47.308124 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":2}
I20260809 19:30:47.309845 12394 raft_consensus.cc:1261] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.309875 12170 raft_consensus.cc:1261] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Refusing update from remote peer 30eebdcbd469457894d1fc8dd5e3af5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:47.310061 12516 consensus_queue.cc:1035] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d284ccc6d14787b845f766d10a9c01" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.310158 12510 consensus_queue.cc:1035] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66f1643b0c1c46d8875f0fd9ffbf2214" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:47.320166 11685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.320257 11685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:30:47.320779 11685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be1cae70fba45eabfc5cdad517cfa27: Sending DeleteTablet for 3 replicas of tablet ba6a6f146c0c4a189580938b3ca38cd5
I20260809 19:30:47.321020 12365 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42450
I20260809 19:30:47.321080 12130 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:48436
I20260809 19:30:47.321115 12551 tablet_replica.cc:330] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: stopping tablet replica
I20260809 19:30:47.321115 11896 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6a6f146c0c4a189580938b3ca38cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:30:47 UTC) from {username='oryx', principal='oryx/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:58232
I20260809 19:30:47.321162 12551 raft_consensus.cc:2229] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.321201 12551 raft_consensus.cc:2258] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.321194 12548 tablet_replica.cc:330] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: stopping tablet replica
I20260809 19:30:47.321208 12547 tablet_replica.cc:330] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c: stopping tablet replica
I20260809 19:30:47.321245 12548 raft_consensus.cc:2229] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:30:47.321295 12547 raft_consensus.cc:2229] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:30:47.321310 12548 raft_consensus.cc:2258] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.321353 12547 raft_consensus.cc:2258] T ba6a6f146c0c4a189580938b3ca38cd5 P 30eebdcbd469457894d1fc8dd5e3af5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:30:47.321422 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 11807
I20260809 19:30:47.338568 12551 ts_tablet_manager.cc:1905] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.340924 12551 ts_tablet_manager.cc:1918] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:47.341014 12551 log.cc:1196] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/ba6a6f146c0c4a189580938b3ca38cd5
I20260809 19:30:47.341393 12551 ts_tablet_manager.cc:1939] T ba6a6f146c0c4a189580938b3ca38cd5 P e9d284ccc6d14787b845f766d10a9c01: Deleting consensus metadata
I20260809 19:30:47.341841 11660 catalog_manager.cc:4841] TS e9d284ccc6d14787b845f766d10a9c01 (127.31.57.195:34961): tablet ba6a6f146c0c4a189580938b3ca38cd5 (table test-table [id=26de0cff89c84094a1147b40edf2851b]) successfully deleted
I20260809 19:30:47.346199 12548 ts_tablet_manager.cc:1905] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:30:47.348124 12548 ts_tablet_manager.cc:1918] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:30:47.348263 12548 log.cc:1196] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/ba6a6f146c0c4a189580938b3ca38cd5
W20260809 19:30:47.348642 11661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:40531 (error 108)
I20260809 19:30:47.348681 12548 ts_tablet_manager.cc:1939] T ba6a6f146c0c4a189580938b3ca38cd5 P 66f1643b0c1c46d8875f0fd9ffbf2214: Deleting consensus metadata
I20260809 19:30:47.349218 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 12023
I20260809 19:30:47.349339 11662 catalog_manager.cc:4841] TS 66f1643b0c1c46d8875f0fd9ffbf2214 (127.31.57.194:39269): tablet ba6a6f146c0c4a189580938b3ca38cd5 (table test-table [id=26de0cff89c84094a1147b40edf2851b]) successfully deleted
I20260809 19:30:47.352063 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:47.349452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:40531 (local address 127.0.0.1:58262)
0809 19:30:47.351919 (+ 2467us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:47.352012 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:40531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2419,"thread_start_us":62,"threads_started":1}
W20260809 19:30:47.352236 11661 catalog_manager.cc:4584] TS 30eebdcbd469457894d1fc8dd5e3af5c (127.31.57.193:40531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba6a6f146c0c4a189580938b3ca38cd5: Network error: Client connection negotiation failed: client connection to 127.31.57.193:40531: connect: Connection refused (error 111)
I20260809 19:30:47.366612 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 12256
I20260809 19:30:47.389223 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 11636
2026-08-09T19:30:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2787 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[12648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[12648](info): set up 2 sockets
Aug 09 19:30:47 dist-test-slave-x4qh krb5kdc[12648](info): commencing operation
krb5kdc: starting...
W20260809 19:30:48.716662 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:48 dist-test-slave-x4qh krb5kdc[12648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:48.823331 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:38425
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:38425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:48.899267 12668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:48.899482 12668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:48.899518 12668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:48.900542 12668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:48.900599 12668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:48.900622 12668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:48.900651 12668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:48.900693 12668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:48.902029 12668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:38425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:38425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:48.902261 12668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:48.902451 12668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:48.906883 12674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:48.906893 12675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:48.906939 12679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:48.907095 12668 server_base.cc:1034] running on GCE node
I20260809 19:30:48.907334 12668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:48.907600 12668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:48.908751 12668 hybrid_clock.cc:648] HybridClock initialized: now 1786303848908734 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T19:30:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1498 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:48 dist-test-slave-x4qh krb5kdc[12686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:48 dist-test-slave-x4qh krb5kdc[12686](info): set up 2 sockets
Aug 09 19:30:48 dist-test-slave-x4qh krb5kdc[12686](info): commencing operation
krb5kdc: starting...
W20260809 19:30:50.066450 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:30:50.174904 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41553
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:50.249863 12708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:50.250065 12708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:50.250100 12708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:50.251106 12708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:50.251163 12708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:50.251183 12708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:50.251200 12708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:50.251224 12708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:50.252591 12708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:50.252833 12708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:50.253038 12708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:50.257649 12722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.257686 12717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.257804 12708 server_base.cc:1034] running on GCE node
W20260809 19:30:50.257664 12715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.258134 12708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:50.258450 12708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:50.259583 12708 hybrid_clock.cc:648] HybridClock initialized: now 1786303850259558 us; error 42 us; skew 500 ppm
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:50.265760 12708 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:50.266177 12708 webserver.cc:460] Webserver started at http://127.31.57.254:39901/ using document root <none> and password file <none>
I20260809 19:30:50.266430 12708 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:50.266494 12708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:50.266623 12708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:50.267621 12708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "790eaca576c442fe84940cbe986064ba"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "c9ef37dc0b10b1f901816c5aa089b483"
server_key_iv: "d8c4c6abecebabc7fa3e444375ece161"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.267951 12708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "790eaca576c442fe84940cbe986064ba"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "c9ef37dc0b10b1f901816c5aa089b483"
server_key_iv: "d8c4c6abecebabc7fa3e444375ece161"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.269176 12708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:50.269840 12730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.269992 12708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:50.270089 12708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "790eaca576c442fe84940cbe986064ba"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "c9ef37dc0b10b1f901816c5aa089b483"
server_key_iv: "d8c4c6abecebabc7fa3e444375ece161"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.270154 12708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:50.299119 12708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:50.301379 12708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:50.301553 12708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:50.305451 12708 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41553
I20260809 19:30:50.305490 12810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41553 every 8 connection(s)
I20260809 19:30:50.305817 12708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:50.306667 12811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:50.308756 12811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Bootstrap starting.
I20260809 19:30:50.309224 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 12708
I20260809 19:30:50.309360 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:30:50.309463 12811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:50.309551 31975 external_mini_cluster.cc:1428] Setting key e3c51df6213a9bd32bab46708aa39ea9
I20260809 19:30:50.309784 12811 log.cc:824] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:50.310573 12811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: No bootstrap required, opened a new log
I20260809 19:30:50.312978 12811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:50.313130 12811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:30:50.313166 12811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790eaca576c442fe84940cbe986064ba, State: Initialized, Role: FOLLOWER
I20260809 19:30:50.313297 12811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:50.313372 12811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:50.313403 12811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:50.313453 12811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:50.314004 12811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:50.314106 12811 leader_election.cc:302] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 790eaca576c442fe84940cbe986064ba; no voters:
I20260809 19:30:50.314262 12811 leader_election.cc:288] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:30:50.314321 12820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:50.314502 12820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 LEADER]: Becoming Leader. State: Replica: 790eaca576c442fe84940cbe986064ba, State: Running, Role: LEADER
I20260809 19:30:50.314507 12811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:50.314636 12820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:50.315062 12821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "790eaca576c442fe84940cbe986064ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } } }
I20260809 19:30:50.315163 12821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:50.315451 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790eaca576c442fe84940cbe986064ba. Latest consensus state: current_term: 1 leader_uuid: "790eaca576c442fe84940cbe986064ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } } }
I20260809 19:30:50.315585 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:50.315748 12833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:50.316234 12833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:50.317696 12833 catalog_manager.cc:1347] Generated new cluster ID: 8ff4be7f84ef409790e2a6c33874e042
I20260809 19:30:50.317745 12833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:50.321193 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.310062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.31.57.254:41553)
0809 19:30:50.310233 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:50.310242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:50.310292 (+ 50us) server_negotiation.cc:407] Connection header received
0809 19:30:50.310898 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:50.310908 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:50.310968 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:50.311109 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:50.311772 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.313136 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.313960 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.314565 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.315026 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.317473 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:50.317477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:50.317477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:50.317487 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:50.319316 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.319717 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.319719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.319720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.319827 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.320030 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.320031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.320031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.320302 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:50.320446 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:50.320878 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 19:30:50.321001 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260809 19:30:50.323706 12833 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:50.324405 12833 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:50.330392 12833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Generated new TSK 0
I20260809 19:30:50.330577 12833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:30:50.359390 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:50.436487 12851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:50.436695 12851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:50.436733 12851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:50.437851 12851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:50.437913 12851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:50.437955 12851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:50.439407 12851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:50.439663 12851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:50.439872 12851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:50.444458 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.444473 12859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.444542 12851 server_base.cc:1034] running on GCE node
W20260809 19:30:50.444478 12860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.444901 12851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:50.445137 12851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:50.446291 12851 hybrid_clock.cc:648] HybridClock initialized: now 1786303850446272 us; error 48 us; skew 500 ppm
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:50.448738 12851 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:30:50.449137 12851 webserver.cc:460] Webserver started at http://127.31.57.193:37755/ using document root <none> and password file <none>
I20260809 19:30:50.449376 12851 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:50.449440 12851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:50.449575 12851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:50.450479 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "6db90a3544764630800c712d8f84b237"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "d3175fdf267ae658c7937c5e8ad56498"
server_key_iv: "aaccd9c77f402ee27455d58fd7238651"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.450811 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "6db90a3544764630800c712d8f84b237"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "d3175fdf267ae658c7937c5e8ad56498"
server_key_iv: "aaccd9c77f402ee27455d58fd7238651"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.452062 12851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:50.452729 12872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.452893 12851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:30:50.452986 12851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "6db90a3544764630800c712d8f84b237"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "d3175fdf267ae658c7937c5e8ad56498"
server_key_iv: "aaccd9c77f402ee27455d58fd7238651"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.453063 12851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:50.471710 12851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:50.473317 12851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:50.473452 12851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:50.473646 12851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:50.473915 12851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:50.473963 12851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.473991 12851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:50.474016 12851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.479488 12851 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:41105
I20260809 19:30:50.479557 13043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:41105 every 8 connection(s)
I20260809 19:30:50.479817 12851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:50.484082 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 12851
I20260809 19:30:50.484177 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:30:50.484365 31975 external_mini_cluster.cc:1428] Setting key f93d75f50c50cc72edb95674a0ff4eb2
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:50.488631 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.480471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59823 (local address 127.31.57.254:41553)
0809 19:30:50.480523 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:50.480525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:50.480686 (+ 161us) server_negotiation.cc:407] Connection header received
0809 19:30:50.481127 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:50.481129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:50.481141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:50.481188 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:50.482011 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.482464 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.483356 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.483590 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.483663 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.485886 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:50.485888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:50.485888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:50.485894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:50.487308 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.487737 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.487739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.487739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.487815 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.488063 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.488064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.488065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.488164 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:50.488226 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:50.488473 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 19:30:50.488553 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:30:50.488755 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.480439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:59823)
0809 19:30:50.480583 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:50.480617 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:50.481033 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:50.481247 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:50.481251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:50.481750 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:50.481914 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.481918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.482552 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.482553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.483258 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.483260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.483671 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.483672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.483786 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.484256 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:50.484267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:50.485801 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:50.487413 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.487420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.487423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.487645 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.487895 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.487896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.487896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.488000 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.488228 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:50.488234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:50.488355 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 19:30:50.488441 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:30:50.488570 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 19:30:50.489151 13044 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41553
I20260809 19:30:50.489244 13044 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:50.489420 13044 heartbeater.cc:511] Master 127.31.57.254:41553 requested a full tablet report, sending...
I20260809 19:30:50.489878 12757 ts_manager.cc:194] Registered new tserver with Master: 6db90a3544764630800c712d8f84b237 (127.31.57.193:41105)
I20260809 19:30:50.490422 12757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:59823
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:30:50.507555 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:50.580953 13050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:50.581104 13050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:50.581123 13050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:50.582152 13050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:50.582190 13050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:50.582235 13050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:50.583634 13050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:50.583853 13050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:50.584069 13050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:50.588616 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.588615 13061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.588706 13060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.588759 13050 server_base.cc:1034] running on GCE node
I20260809 19:30:50.589068 13050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:50.589299 13050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:50.590431 13050 hybrid_clock.cc:648] HybridClock initialized: now 1786303850590413 us; error 34 us; skew 500 ppm
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:50.592669 13050 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:30:50.593083 13050 webserver.cc:460] Webserver started at http://127.31.57.194:36253/ using document root <none> and password file <none>
I20260809 19:30:50.593281 13050 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:50.593345 13050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:50.593487 13050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:50.594485 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "822f0de1cf1343deb185d971ed6e3a8f"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "867b209c3019d28643072a6aa8a61911"
server_key_iv: "512c108251658a2316f0842b0bfef8d3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.594851 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "822f0de1cf1343deb185d971ed6e3a8f"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "867b209c3019d28643072a6aa8a61911"
server_key_iv: "512c108251658a2316f0842b0bfef8d3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.596216 13050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:30:50.596873 13078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.597036 13050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.597149 13050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "822f0de1cf1343deb185d971ed6e3a8f"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "867b209c3019d28643072a6aa8a61911"
server_key_iv: "512c108251658a2316f0842b0bfef8d3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.597216 13050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:50.612893 13050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:50.614904 13050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:50.615054 13050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:50.615288 13050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:50.615664 13050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:50.615707 13050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.615734 13050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:50.615767 13050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.621635 13050 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:33577
I20260809 19:30:50.621680 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:33577 every 8 connection(s)
I20260809 19:30:50.621956 13050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:50.630218 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.622518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:57403 (local address 127.31.57.254:41553)
0809 19:30:50.622570 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:50.622572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:50.622760 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:30:50.623234 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:50.623236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:50.623250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:50.623307 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:50.624195 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.624696 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.625643 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.625864 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.625934 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.627822 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:50.627824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:50.627824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:50.627830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:50.628954 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.629272 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.629272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.629273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.629320 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.629603 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.629605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.629605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.629708 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:50.629789 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:50.630061 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 19:30:50.630135 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:50.630388 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.622478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:57403)
0809 19:30:50.622624 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:50.622664 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:50.623132 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:50.623387 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:50.623391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:50.623931 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:50.624099 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.624103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.624791 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.624793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.625550 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.625553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.625958 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.625959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.626075 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.626459 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:50.626467 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:50.627734 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:50.629019 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.629025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.629026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.629213 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.629418 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.629419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.629420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.629530 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.629772 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:50.629779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:50.629939 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 19:30:50.630038 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:30:50.630177 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260809 19:30:50.630802 13244 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41553
I20260809 19:30:50.630900 13244 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:50.631052 13244 heartbeater.cc:511] Master 127.31.57.254:41553 requested a full tablet report, sending...
I20260809 19:30:50.631417 12757 ts_manager.cc:194] Registered new tserver with Master: 822f0de1cf1343deb185d971ed6e3a8f (127.31.57.194:33577)
I20260809 19:30:50.631556 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 13050
I20260809 19:30:50.631656 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:30:50.631821 31975 external_mini_cluster.cc:1428] Setting key ac510ab61a33f8ac692d0040828c333b
I20260809 19:30:50.631845 12757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:57403
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:30:50.655146 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:50.731045 13250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:50.731202 13250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:50.731223 13250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:50.732265 13250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:50.732311 13250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:50.732358 13250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:50.733577 13250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:50.733773 13250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:50.733958 13250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:50.738492 13259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.738519 13261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.738519 13250 server_base.cc:1034] running on GCE node
W20260809 19:30:50.738672 13263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.738885 13250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:50.739094 13250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:50.740236 13250 hybrid_clock.cc:648] HybridClock initialized: now 1786303850740222 us; error 25 us; skew 500 ppm
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:50.742596 13250 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:30:50.742939 13250 webserver.cc:460] Webserver started at http://127.31.57.195:32799/ using document root <none> and password file <none>
I20260809 19:30:50.743099 13250 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:50.743137 13250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:50.743216 13250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:50.744135 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "97a5581b79d94197a9310478a0bf94a7"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "a35277f6a26f352a3efcb1e4e55c9d8f"
server_key_iv: "9a7fb8a24aac18f075de02102cc1bff3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.744428 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "97a5581b79d94197a9310478a0bf94a7"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "a35277f6a26f352a3efcb1e4e55c9d8f"
server_key_iv: "9a7fb8a24aac18f075de02102cc1bff3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.745643 13250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:50.746253 13273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.746397 13250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:50.746474 13250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "97a5581b79d94197a9310478a0bf94a7"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "a35277f6a26f352a3efcb1e4e55c9d8f"
server_key_iv: "9a7fb8a24aac18f075de02102cc1bff3"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.746522 13250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:50.756342 13250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:50.757901 13250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:50.757998 13250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:50.758147 13250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:50.758364 13250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:50.758392 13250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.758410 13250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:50.758421 13250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.763546 13250 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:43935
I20260809 19:30:50.763612 13465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:43935 every 8 connection(s)
I20260809 19:30:50.763885 13250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:30:50.769044 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 13250
I20260809 19:30:50.769142 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:30:50.769326 31975 external_mini_cluster.cc:1428] Setting key 89785ddc88451f0014d69bcecf76b7a5
I20260809 19:30:50.772176 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.764583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:37091 (local address 127.31.57.254:41553)
0809 19:30:50.764625 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:50.764627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:50.764806 (+ 179us) server_negotiation.cc:407] Connection header received
0809 19:30:50.765214 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:50.765215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:50.765224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:50.765261 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:50.766054 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.766495 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.767322 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.767586 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.767634 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.769521 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:50.769522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:50.769523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:50.769544 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:50.770884 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.771273 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.771274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.771275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.771340 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:50.771604 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:50.771605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:50.771606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:50.771712 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:50.771777 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:50.772018 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 19:30:50.772092 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:30:50.772390 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:50.764541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:37091)
0809 19:30:50.764688 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:50.764726 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:50.765116 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:50.765322 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:50.765324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:50.765822 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:50.765993 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.765996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.766570 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.766571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.767247 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:50.767249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:50.767639 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:50.767640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:50.767734 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:50.768073 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:50.768080 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:50.769432 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:50.770978 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.770984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.770986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.771203 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.771434 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:50.771435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:50.771436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:50.771533 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:50.771769 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:50.771774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:50.771898 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 19:30:50.771985 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:30:50.772142 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260809 19:30:50.772759 13467 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41553
I20260809 19:30:50.772827 13467 heartbeater.cc:464] Registering TS with master...
I20260809 19:30:50.772938 13467 heartbeater.cc:511] Master 127.31.57.254:41553 requested a full tablet report, sending...
I20260809 19:30:50.773226 12757 ts_manager.cc:194] Registered new tserver with Master: 97a5581b79d94197a9310478a0bf94a7 (127.31.57.195:43935)
I20260809 19:30:50.773574 12757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:37091
I20260809 19:30:50.780718 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
Entry for principal oryx/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab.
WARNING: no policy specified for oryx/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
Entry for principal oryx/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab.
WARNING: no policy specified for oryx/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
Entry for principal oryx/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab.
I20260809 19:30:50.843729 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 12851
I20260809 19:30:50.854210 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 13050
I20260809 19:30:50.861045 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 13250
I20260809 19:30:50.873383 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 12708
I20260809 19:30:50.882494 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41553
--webserver_interface=127.31.57.254
--webserver_port=39901
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:50.960237 13482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:50.960412 13482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:50.960431 13482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:50.961419 13482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:50.961452 13482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:50.961465 13482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:50.961477 13482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:50.961488 13482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:50.962795 13482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=39901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:50.962956 13482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:50.963099 13482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:50.967558 13493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:50.967686 13492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.967657 13482 server_base.cc:1034] running on GCE node
W20260809 19:30:50.967835 13495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:50.968040 13482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:50.968297 13482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:50.969451 13482 hybrid_clock.cc:648] HybridClock initialized: now 1786303850969444 us; error 33 us; skew 500 ppm
Aug 09 19:30:50 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303850, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:50.971760 13482 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:30:50.972190 13482 webserver.cc:460] Webserver started at http://127.31.57.254:39901/ using document root <none> and password file <none>
I20260809 19:30:50.972399 13482 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:50.972477 13482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:50.973712 13482 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:50.974287 13505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.974442 13482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:50.974534 13482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "790eaca576c442fe84940cbe986064ba"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "c9ef37dc0b10b1f901816c5aa089b483"
server_key_iv: "d8c4c6abecebabc7fa3e444375ece161"
server_key_version: "encryptionkey@0"
I20260809 19:30:50.974843 13482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:50.987479 13482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:50.989156 13482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:50.989421 13482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:50.993673 13482 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41553
I20260809 19:30:50.993701 13591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41553 every 8 connection(s)
I20260809 19:30:50.994076 13482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:50.995457 13593 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:30:50.996061 13593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Bootstrap starting.
I20260809 19:30:50.996884 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 13482
I20260809 19:30:50.997202 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:41105
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:41553
--webserver_port=37755
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:30:51.004669 13593 log.cc:824] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:51.006137 13593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:30:51.006400 13593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Bootstrap complete.
I20260809 19:30:51.008548 13593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:51.008790 13593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790eaca576c442fe84940cbe986064ba, State: Initialized, Role: FOLLOWER
I20260809 19:30:51.008970 13593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:51.009052 13593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:30:51.009100 13593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:30:51.009143 13593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:30:51.010071 13593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:51.010180 13593 leader_election.cc:302] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 790eaca576c442fe84940cbe986064ba; no voters:
I20260809 19:30:51.010336 13593 leader_election.cc:288] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:30:51.010382 13600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:30:51.010535 13600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [term 2 LEADER]: Becoming Leader. State: Replica: 790eaca576c442fe84940cbe986064ba, State: Running, Role: LEADER
I20260809 19:30:51.010572 13593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:51.010627 13600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } }
I20260809 19:30:51.010891 13600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790eaca576c442fe84940cbe986064ba. Latest consensus state: current_term: 2 leader_uuid: "790eaca576c442fe84940cbe986064ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } } }
I20260809 19:30:51.010958 13600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:51.010895 13602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "790eaca576c442fe84940cbe986064ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790eaca576c442fe84940cbe986064ba" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41553 } } }
I20260809 19:30:51.011149 13602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:51.011541 13612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:51.012055 13612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:51.012177 13612 catalog_manager.cc:1259] Loaded cluster ID: 8ff4be7f84ef409790e2a6c33874e042
I20260809 19:30:51.012210 13612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:51.013223 13612 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:51.013399 13612 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 790eaca576c442fe84940cbe986064ba: Loaded TSK: 0
I20260809 19:30:51.013603 13612 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:30:51.078162 13597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:51.078418 13597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:51.078460 13597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:51.079671 13597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:51.079727 13597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:51.079768 13597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:30:51.081137 13597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.193.keytab
--principal=oryx/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:41105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=37755
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:51.081390 13597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:51.081596 13597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:51.086874 13640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:51.086921 13638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:51.086939 13645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:51.087266 13597 server_base.cc:1034] running on GCE node
I20260809 19:30:51.087484 13597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:51.087740 13597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:51.088910 13597 hybrid_clock.cc:648] HybridClock initialized: now 1786303851088899 us; error 25 us; skew 500 ppm
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303851, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:51.091655 13597 init.cc:377] Logged in from keytab as oryx/127.31.57.193@KRBTEST.COM (short username oryx)
I20260809 19:30:51.092074 13597 webserver.cc:460] Webserver started at http://127.31.57.193:37755/ using document root <none> and password file <none>
I20260809 19:30:51.092314 13597 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:51.092408 13597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:51.094877 13597 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 19:30:51.095546 13654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.095736 13597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:30:51.095832 13597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "6db90a3544764630800c712d8f84b237"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "d3175fdf267ae658c7937c5e8ad56498"
server_key_iv: "aaccd9c77f402ee27455d58fd7238651"
server_key_version: "encryptionkey@0"
I20260809 19:30:51.096180 13597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:51.102942 13597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:51.104961 13597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:51.105110 13597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:51.105324 13597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:51.105604 13597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:51.105650 13597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.105715 13597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:51.105746 13597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.111568 13597 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:41105
I20260809 19:30:51.111611 13848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:41105 every 8 connection(s)
I20260809 19:30:51.112001 13597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:30:51.112833 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 13597
I20260809 19:30:51.113090 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:33577
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:41553
--webserver_port=36253
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.120286 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.112685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:39355)
0809 19:30:51.113013 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.113051 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.113638 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.114081 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.114085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.114778 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.115023 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.115029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.116410 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.116412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.117085 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.117087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.117679 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.117680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.117798 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.118204 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.118211 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.120109 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":84,"threads_started":1}
W20260809 19:30:51.120422 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.112742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:39355 (local address 127.31.57.254:41553)
0809 19:30:51.112930 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.112937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.113373 (+ 436us) server_negotiation.cc:407] Connection header received
0809 19:30:51.113847 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.113853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.113901 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.114019 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.115156 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.116319 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.117176 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.117598 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.117699 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.120243 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:39355: BlockingRecv error: recv got EOF from 127.31.57.193:39355 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
W20260809 19:30:51.120676 13849 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41553 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.57.254:41553: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.124524 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.120889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:49159)
0809 19:30:51.120924 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.120933 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.120965 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.121073 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.121073 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.121185 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.121225 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.121227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.121759 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.121760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.122199 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.122201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.122519 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.122520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.122578 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.122927 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.122932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.124455 (+ 1523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:30:51.124575 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.120936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:49159 (local address 127.31.57.254:41553)
0809 19:30:51.120967 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.120968 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.120975 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:51.121012 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.121013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.121022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.121070 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.121291 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.121670 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.122275 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.122456 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.122519 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.124521 (+ 2002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:49159: BlockingRecv error: recv got EOF from 127.31.57.193:49159 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.128504 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.124994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:38215)
0809 19:30:51.125028 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.125037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.125072 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.125194 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.125195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.125317 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.125338 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.125339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.125883 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.125883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.126285 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.126286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.126583 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.126584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.126629 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.126970 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.126975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.128442 (+ 1467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:30:51.128546 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.125038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:38215 (local address 127.31.57.254:41553)
0809 19:30:51.125069 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.125070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.125079 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:51.125129 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.125130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.125139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.125190 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.125417 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.125816 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.126354 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.126523 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.126584 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.128498 (+ 1914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:38215: BlockingRecv error: recv got EOF from 127.31.57.193:38215 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:30:51.128821 13849 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:30:51.194839 13857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:51.195019 13857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:51.195041 13857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:51.196123 13857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:51.196163 13857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:51.196187 13857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:30:51.197429 13857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.194.keytab
--principal=oryx/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:33577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=36253
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:51.197585 13857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:51.197733 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:51.202306 13866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:51.202346 13867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:51.202335 13857 server_base.cc:1034] running on GCE node
W20260809 19:30:51.202316 13871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:51.202685 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:51.202914 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:51.204061 13857 hybrid_clock.cc:648] HybridClock initialized: now 1786303851204047 us; error 26 us; skew 500 ppm
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303851, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:51.206298 13857 init.cc:377] Logged in from keytab as oryx/127.31.57.194@KRBTEST.COM (short username oryx)
I20260809 19:30:51.206717 13857 webserver.cc:460] Webserver started at http://127.31.57.194:36253/ using document root <none> and password file <none>
I20260809 19:30:51.206912 13857 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:51.206987 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:51.208249 13857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:51.208747 13884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.208891 13857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.208973 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "822f0de1cf1343deb185d971ed6e3a8f"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "867b209c3019d28643072a6aa8a61911"
server_key_iv: "512c108251658a2316f0842b0bfef8d3"
server_key_version: "encryptionkey@0"
I20260809 19:30:51.209241 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:51.234907 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:51.236488 13857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:51.236631 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:51.236833 13857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:51.237123 13857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:51.237190 13857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.237233 13857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:51.237259 13857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.243439 13857 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:33577
I20260809 19:30:51.243491 14074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:33577 every 8 connection(s)
I20260809 19:30:51.243793 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.249184 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 13857
I20260809 19:30:51.249430 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:43935
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:41553
--webserver_port=32799
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.250223 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.244429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:49849)
0809 19:30:51.244596 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.244628 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.245029 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.245216 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.245219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.245723 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.245896 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.245901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.246465 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.246465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.247244 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.247246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.247629 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.247630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.247729 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.248098 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.248105 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.250112 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
W20260809 19:30:51.250278 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.244429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49849 (local address 127.31.57.254:41553)
0809 19:30:51.244484 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.244487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.244709 (+ 222us) server_negotiation.cc:407] Connection header received
0809 19:30:51.245105 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.245106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.245117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.245167 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.245962 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.246395 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.247315 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.247564 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.247614 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.250208 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:49849: BlockingRecv error: recv got EOF from 127.31.57.194:49849 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 19:30:51.250469 14077 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41553 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.57.254:41553: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.255139 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.250714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:58735)
0809 19:30:51.250749 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.250757 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.250795 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.250893 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.250894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.251053 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.251118 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.251119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.251887 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.251888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.252528 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.252531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.252862 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.252863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.252938 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.253299 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.253304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.255051 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:30:51.255160 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.250758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:58735 (local address 127.31.57.254:41553)
0809 19:30:51.250793 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.250795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.250803 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:51.250832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.250833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.250839 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.250880 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.251205 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.251820 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.252618 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.252795 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.252844 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.255121 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:58735: BlockingRecv error: recv got EOF from 127.31.57.194:58735 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.259338 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.255406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:38279)
0809 19:30:51.255443 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.255452 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.255490 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.255630 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.255631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.255784 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.255828 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.255841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.256348 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.256349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.257116 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.257118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.257401 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.257402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.257457 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.257736 (+ 279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.257741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.259273 (+ 1532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:30:51.259421 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.255435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:38279 (local address 127.31.57.254:41553)
0809 19:30:51.255468 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.255469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.255478 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:51.255558 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.255559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.255566 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.255604 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.255903 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.256282 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.257188 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.257336 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.257381 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.259334 (+ 1953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:38279: BlockingRecv error: recv got EOF from 127.31.57.194:38279 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:30:51.259469 14077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:30:51.326910 14080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:51.327066 14080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:51.327085 14080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:51.328096 14080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:51.328140 14080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:51.328181 14080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:30:51.329450 14080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/krb5kdc/oryx_127.31.57.195.keytab
--principal=oryx/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:43935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=32799
--tserver_master_addrs=127.31.57.254:41553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:51.329640 14080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:51.329830 14080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:51.334292 14090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:51.334291 14088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:51.334364 14080 server_base.cc:1034] running on GCE node
W20260809 19:30:51.334332 14092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:51.334713 14080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:51.334949 14080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:51.336123 14080 hybrid_clock.cc:648] HybridClock initialized: now 1786303851336104 us; error 36 us; skew 500 ppm
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303851, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:51.338361 14080 init.cc:377] Logged in from keytab as oryx/127.31.57.195@KRBTEST.COM (short username oryx)
I20260809 19:30:51.338681 14080 webserver.cc:460] Webserver started at http://127.31.57.195:32799/ using document root <none> and password file <none>
I20260809 19:30:51.338883 14080 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:51.338949 14080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:51.340299 14080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:30:51.340811 14104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.340976 14080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:51.341069 14080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "97a5581b79d94197a9310478a0bf94a7"
format_stamp: "Formatted at 2026-08-09 19:30:50 on dist-test-slave-x4qh"
server_key: "a35277f6a26f352a3efcb1e4e55c9d8f"
server_key_iv: "9a7fb8a24aac18f075de02102cc1bff3"
server_key_version: "encryptionkey@0"
I20260809 19:30:51.341346 14080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:51.382050 14080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:51.384313 14080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:51.384490 14080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:51.384722 14080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:30:51.385028 14080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:30:51.385082 14080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.385123 14080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:30:51.385151 14080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:51.390625 14080 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:43935
I20260809 19:30:51.390689 14305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:43935 every 8 connection(s)
I20260809 19:30:51.391010 14080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:30:51.395867 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 14080
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:30:51.397740 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.391785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:38117 (local address 127.31.57.254:41553)
0809 19:30:51.391844 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.391846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.392048 (+ 202us) server_negotiation.cc:407] Connection header received
0809 19:30:51.392499 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.392500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.392509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.392555 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.393336 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.393765 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.394628 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.394836 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.394887 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.397689 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:38117: BlockingRecv error: recv got EOF from 127.31.57.195:38117 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:30:51.397815 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.391759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:38117)
0809 19:30:51.391920 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.391959 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.392408 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.392609 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.392612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.393099 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.393260 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.393265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.393848 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.393849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.394550 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.394552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.394903 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.394904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.395008 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.395389 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.395396 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.397628 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260809 19:30:51.397979 14306 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41553 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.57.254:41553: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.402467 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.398200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:47655)
0809 19:30:51.398241 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.398250 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.398285 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.398398 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.398399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.398511 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.398546 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.398547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.399281 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.399281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.399875 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.399877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.400243 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.400244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.400318 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.400666 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.400670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.402396 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:30:51.402524 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.398252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:47655 (local address 127.31.57.254:41553)
0809 19:30:51.398308 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.398310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.398318 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:51.398337 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.398337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.398344 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.398394 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.398613 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.399205 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.399950 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.400149 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.400241 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.402459 (+ 2218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:47655: BlockingRecv error: recv got EOF from 127.31.57.195:47655 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:30:51.403685 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.396361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.31.57.254:41553)
0809 19:30:51.396489 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.396491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.396539 (+ 48us) server_negotiation.cc:407] Connection header received
0809 19:30:51.396605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.396606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.396618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.396686 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.397365 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.397810 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.398753 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.398962 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.399034 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.400315 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:51.400319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:51.400319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:51.400330 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:51.402125 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:51.402476 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:51.402479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:51.402480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:51.402617 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:51.402847 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:51.402848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:51.402848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:51.403146 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:51.403295 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:51.403473 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 19:30:51.403583 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:51 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:51.406836 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:51.402690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:50959)
0809 19:30:51.402831 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:51.402843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:51.402890 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:51.402991 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:51.402991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:51.403165 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:51.403225 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.403227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.403967 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.403968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.404728 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:51.404731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.405033 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.405034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:51.405075 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.405398 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:51.405402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:51.406787 (+ 1385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
W20260809 19:30:51.406891 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:51.402850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50959 (local address 127.31.57.254:41553)
0809 19:30:51.402884 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:51.402886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:51.402893 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:51.402940 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:51.402941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:51.402947 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:51.402985 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:51.403295 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.403889 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.404793 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:51.404958 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:51.405010 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:51.406860 (+ 1850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:50959: BlockingRecv error: recv got EOF from 127.31.57.195:50959 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:30:51.407001 14306 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:52.133416 14316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:52.129176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:48015)
0809 19:30:52.129340 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:52.129354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:52.129416 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:52.129616 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:52.129617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:52.129767 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:52.129821 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.129822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.130407 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.130407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.130908 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.130910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.131187 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.131187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.131223 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.131635 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:52.131639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:52.133359 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
W20260809 19:30:52.133546 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:52.129340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:48015 (local address 127.31.57.254:41553)
0809 19:30:52.129465 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:52.129467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:52.129473 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:30:52.129493 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:52.129493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:52.129502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:52.129556 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:52.129881 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.130341 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.130973 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.131131 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.131174 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.133461 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:48015: BlockingRecv error: recv got EOF from 127.31.57.193:48015 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:52.264216 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:52.260004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:35567)
0809 19:30:52.260159 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:52.260170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:52.260231 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:52.260360 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:52.260361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:52.260527 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:52.260570 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.260571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.261165 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.261166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.261637 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.261639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.261962 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.261963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.262014 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.262443 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:52.262449 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:52.264129 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
W20260809 19:30:52.264256 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:52.260086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:35567 (local address 127.31.57.254:41553)
0809 19:30:52.260160 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:52.260162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:52.260216 (+ 54us) server_negotiation.cc:407] Connection header received
0809 19:30:52.260279 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:52.260280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:52.260291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:52.260348 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:52.260642 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.261087 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.261719 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.261894 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.261963 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.264198 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:35567: BlockingRecv error: recv got EOF from 127.31.57.194:35567 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:52 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:52.412063 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:52.407437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:44175)
0809 19:30:52.407624 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:52.407639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:52.407725 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:52.407906 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:52.407907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:52.408072 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:52.408116 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.408118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.408782 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.408783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.409248 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:52.409249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.409604 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.409605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:52.409667 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.410144 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:52.410148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:52.412004 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
W20260809 19:30:52.412113 14317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:52.407509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:44175 (local address 127.31.57.254:41553)
0809 19:30:52.407606 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:52.407609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:52.407706 (+ 97us) server_negotiation.cc:407] Connection header received
0809 19:30:52.407814 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:52.407815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:52.407827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:52.407899 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:52.408192 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.408679 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.409342 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:52.409523 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:52.409603 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:52.412072 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:44175: BlockingRecv error: recv got EOF from 127.31.57.195:44175 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:53.138429 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:53.133867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:48927)
0809 19:30:53.133993 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:53.134002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:53.134048 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:53.134201 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:53.134201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:53.134368 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:53.134418 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.134420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.135028 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.135028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.135694 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.135698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.136197 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.136198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.136244 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.136680 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:53.136684 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:53.138347 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
W20260809 19:30:53.138480 14324 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:53.133954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:48927 (local address 127.31.57.254:41553)
0809 19:30:53.134082 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:53.134083 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:53.134091 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:53.134112 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:53.134112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:53.134121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:53.134183 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:53.134477 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.134964 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.135805 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.136117 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.136183 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.138415 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:48927: BlockingRecv error: recv got EOF from 127.31.57.193:48927 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:53.269109 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:53.264671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:52355)
0809 19:30:53.264871 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:53.264883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:53.264965 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:53.265105 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:53.265106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:53.265295 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:53.265341 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.265343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.266006 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.266007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.266486 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.266488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.266822 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.266823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.266880 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.267280 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:53.267285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:53.269019 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
W20260809 19:30:53.269161 14324 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:53.264772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:52355 (local address 127.31.57.254:41553)
0809 19:30:53.264856 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:53.264859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:53.264951 (+ 92us) server_negotiation.cc:407] Connection header received
0809 19:30:53.265035 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:53.265036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:53.265047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:53.265091 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:53.265419 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.265905 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.266568 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.266749 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.266821 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.269097 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:52355: BlockingRecv error: recv got EOF from 127.31.57.194:52355 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:53 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:53.417107 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:53.412471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:55965)
0809 19:30:53.412605 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:53.412627 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:53.412678 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:53.412798 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:53.412798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:53.413070 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:53.413135 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.413137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.413901 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.413902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.414409 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:53.414410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.414725 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.414726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:53.414762 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.415202 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:53.415206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:53.417057 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
W20260809 19:30:53.417164 14324 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:53.412571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:55965 (local address 127.31.57.254:41553)
0809 19:30:53.412633 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:53.412635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:53.412683 (+ 48us) server_negotiation.cc:407] Connection header received
0809 19:30:53.412719 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:53.412719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:53.412731 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:53.412783 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:53.413239 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.413817 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.414494 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:53.414670 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:53.414715 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:53.417122 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:55965: BlockingRecv error: recv got EOF from 127.31.57.195:55965 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:54.143576 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:54.139075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:39793)
0809 19:30:54.139195 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:54.139209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:54.139279 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:54.139451 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:54.139452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:54.139606 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:54.139656 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.139658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.140266 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.140267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.140747 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.140748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.141034 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.141034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.141071 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.141520 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:54.141524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:54.143523 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
W20260809 19:30:54.143637 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:54.139172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:39793 (local address 127.31.57.254:41553)
0809 19:30:54.139309 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:54.139311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:54.139318 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:54.139340 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:54.139340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:54.139350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:54.139419 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:54.139732 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.140206 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.140806 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.140976 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.141021 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.143586 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:39793: BlockingRecv error: recv got EOF from 127.31.57.193:39793 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:54.274144 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:54.269741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:41669)
0809 19:30:54.269868 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:54.269886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:54.269943 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:54.270051 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:54.270052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:54.270205 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:54.270254 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.270255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.270852 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.270852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.271346 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.271347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.271906 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.271907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.271971 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.272459 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:54.272464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:54.274093 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
W20260809 19:30:54.274269 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:54.269830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:41669 (local address 127.31.57.254:41553)
0809 19:30:54.269882 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:54.269884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:54.269947 (+ 63us) server_negotiation.cc:407] Connection header received
0809 19:30:54.269988 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:54.269989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:54.269999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:54.270041 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:54.270321 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.270787 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.271522 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.271832 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.271900 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.274193 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:41669: BlockingRecv error: recv got EOF from 127.31.57.194:41669 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:54 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:54.422147 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:54.417547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:56093)
0809 19:30:54.417674 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:54.417683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:54.417767 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:54.417953 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:54.417953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:54.418107 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:54.418156 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.418158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.418785 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.418786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.419290 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:54.419292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.419714 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.419715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:54.419758 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.420211 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:54.420216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:54.422098 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260809 19:30:54.422258 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:54.417612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:56093 (local address 127.31.57.254:41553)
0809 19:30:54.417688 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:54.417690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:54.417760 (+ 70us) server_negotiation.cc:407] Connection header received
0809 19:30:54.417869 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:54.417870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:54.417882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:54.417931 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:54.418219 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.418713 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.419360 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:54.419655 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:54.419704 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:54.422171 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:56093: BlockingRecv error: recv got EOF from 127.31.57.195:56093 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:55.148631 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:55.143968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:60707)
0809 19:30:55.144137 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:55.144148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:55.144208 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:55.144384 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:55.144385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:55.144586 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:55.144635 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.144637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.145311 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.145312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.145817 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.145819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.146206 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.146206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.146268 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.146709 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:55.146713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:55.148579 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
W20260809 19:30:55.148731 14335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:55.144083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:60707 (local address 127.31.57.254:41553)
0809 19:30:55.144231 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:55.144234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:55.144240 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:30:55.144262 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:55.144263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:55.144275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:55.144366 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:55.144713 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.145227 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.145902 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.146097 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.146205 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.148661 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:60707: BlockingRecv error: recv got EOF from 127.31.57.193:60707 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:55.282249 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:55.274606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:34453)
0809 19:30:55.274798 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:55.274813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:55.274886 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:55.275042 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:55.275043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:55.275208 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:55.275258 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.275259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.275886 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.275887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.276437 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.276438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.276755 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.276755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.276796 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.280394 (+ 3598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:55.280399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:55.282191 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
W20260809 19:30:55.282369 14335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:55.274775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:34453 (local address 127.31.57.254:41553)
0809 19:30:55.274854 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:55.274855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:55.274863 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:55.274946 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:55.274947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:55.274957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:55.275013 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:55.275324 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.275822 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.276507 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.276690 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.276741 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.282296 (+ 5555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:34453: BlockingRecv error: recv got EOF from 127.31.57.194:34453 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:55 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:55.427095 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:55.422668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:37615)
0809 19:30:55.422832 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:55.422845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:55.422909 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:55.423064 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:55.423065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:55.423305 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:55.423403 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.423405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.424133 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.424133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.424617 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:55.424618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.424901 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.424902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:55.424938 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.425364 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:55.425368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:55.427025 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":59,"threads_started":1}
W20260809 19:30:55.427157 14335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:55.422884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:37615 (local address 127.31.57.254:41553)
0809 19:30:55.422950 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:55.422952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:55.422959 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:55.422981 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:55.422982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:55.422992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:55.423040 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:55.423474 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.424068 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.424682 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:55.424845 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:55.424890 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:55.427112 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:37615: BlockingRecv error: recv got EOF from 127.31.57.195:37615 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:56.153749 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:56.149166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:54507)
0809 19:30:56.149345 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:56.149355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:56.149410 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:56.149541 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:56.149542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:56.149701 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:56.149752 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.149754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.150369 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.150369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.150862 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.150864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.151418 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.151419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.151477 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.151909 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:56.151913 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:56.153694 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
W20260809 19:30:56.153868 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:56.149259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:54507 (local address 127.31.57.254:41553)
0809 19:30:56.149384 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:56.149386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:56.149409 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:30:56.149466 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:56.149466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:56.149476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:56.149524 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:56.149819 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.150305 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.150947 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.151307 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.151373 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.153781 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:54507: BlockingRecv error: recv got EOF from 127.31.57.193:54507 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:56.287650 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:56.282815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:58417)
0809 19:30:56.282978 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:56.282990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:56.283069 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:56.283261 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:56.283262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:56.283476 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:56.283528 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.283530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.284219 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.284220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.284795 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.284797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.285213 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.285214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.285287 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.285716 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:56.285720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:56.287595 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":55,"threads_started":1}
W20260809 19:30:56.287791 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:56.282915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:58417 (local address 127.31.57.254:41553)
0809 19:30:56.283009 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:56.283011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:56.283055 (+ 44us) server_negotiation.cc:407] Connection header received
0809 19:30:56.283156 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:56.283157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:56.283168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:56.283251 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:56.283612 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.284131 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.284901 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.285122 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.285210 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.287702 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:58417: BlockingRecv error: recv got EOF from 127.31.57.194:58417 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:56 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:56.431926 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:56.427571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:54281)
0809 19:30:56.427719 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:56.427729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:56.427813 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:56.427966 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:56.427967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:56.428122 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:56.428171 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.428173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.428767 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.428767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.429278 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:56.429279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.429583 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.429584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:56.429623 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.430056 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:56.430060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:56.431870 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
W20260809 19:30:56.432052 14340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:56.427605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54281 (local address 127.31.57.254:41553)
0809 19:30:56.427671 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:56.427674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:56.427795 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:30:56.427878 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:56.427879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:56.427892 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:56.427945 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:56.428230 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.428705 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.429354 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:56.429524 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:56.429571 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:56.431972 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:54281: BlockingRecv error: recv got EOF from 127.31.57.195:54281 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:57.158607 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:57.154272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:50733)
0809 19:30:57.154407 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:57.154422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:57.154469 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:57.154636 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:57.154637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:57.154796 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:57.154847 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.154848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.155514 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.155515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.156025 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.156026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.156358 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.156359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.156398 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.156826 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:57.156831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:57.158551 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260809 19:30:57.158728 14345 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:57.154361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:50733 (local address 127.31.57.254:41553)
0809 19:30:57.154501 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:57.154503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:57.154511 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:57.154542 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:57.154543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:57.154553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:57.154616 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:57.154909 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.155449 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.156099 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.156284 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.156334 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.158641 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:50733: BlockingRecv error: recv got EOF from 127.31.57.193:50733 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:57.292810 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:57.288172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:53367)
0809 19:30:57.288346 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:57.288357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:57.288429 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:57.288560 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:57.288561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:57.288748 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:57.288799 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.288801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.289706 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.289707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.290221 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.290223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.290528 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.290529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.290569 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.290980 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:57.290984 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:57.292718 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
W20260809 19:30:57.292869 14345 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:57.288247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:53367 (local address 127.31.57.254:41553)
0809 19:30:57.288317 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:57.288320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:57.288431 (+ 111us) server_negotiation.cc:407] Connection header received
0809 19:30:57.288488 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:57.288489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:57.288499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:57.288540 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:57.288870 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.289624 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.290291 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.290465 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.290515 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.292810 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:53367: BlockingRecv error: recv got EOF from 127.31.57.194:53367 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:57 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:57.436574 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:57.432354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:35901)
0809 19:30:57.432499 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:57.432509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:57.432566 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:57.432692 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:57.432693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:57.432871 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:57.432919 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.432921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.433491 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.433492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.433996 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:57.433997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.434312 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.434313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:57.434352 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.434790 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:57.434794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:57.436517 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
W20260809 19:30:57.436668 14345 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:57.432416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:35901 (local address 127.31.57.254:41553)
0809 19:30:57.432488 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:57.432490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:57.432568 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:30:57.432607 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:57.432607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:57.432619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:57.432670 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:57.432980 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.433430 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.434076 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:57.434250 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:57.434298 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:57.436603 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:35901: BlockingRecv error: recv got EOF from 127.31.57.195:35901 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:58.167932 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:58.159297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:41469)
0809 19:30:58.159415 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:58.159427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:58.159486 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:58.159646 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:58.159646 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:58.159849 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:58.159913 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.159915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.160733 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.160733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.161234 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.161236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.161551 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.161552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.161589 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.165984 (+ 4395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:58.165991 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:58.167833 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":88,"threads_started":1}
W20260809 19:30:58.167986 14351 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:58.159351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:41469 (local address 127.31.57.254:41553)
0809 19:30:58.159520 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:58.159522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:58.159530 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:58.159551 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:58.159552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:58.159562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:58.159625 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:58.159982 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.160665 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.161310 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.161490 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.161539 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.167917 (+ 6378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:41469: BlockingRecv error: recv got EOF from 127.31.57.193:41469 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:58.297456 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:58.293251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:57353)
0809 19:30:58.293406 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:58.293422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:58.293485 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:58.293620 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:58.293620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:58.293795 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:58.293843 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.293845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.294412 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.294412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.294876 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.294877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.295154 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.295155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.295191 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.295719 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:58.295723 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:58.297402 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":59,"threads_started":1}
W20260809 19:30:58.297562 14351 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:58.293358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:57353 (local address 127.31.57.254:41553)
0809 19:30:58.293436 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:58.293438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:58.293468 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:30:58.293542 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:58.293543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:58.293553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:58.293594 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:58.293902 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.294352 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.294939 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.295098 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.295143 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.297492 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:57353: BlockingRecv error: recv got EOF from 127.31.57.194:57353 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:58 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:58.441854 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:58.437028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:43537)
0809 19:30:58.437155 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:58.437166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:58.437239 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:58.437404 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:58.437405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:58.437584 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:58.437631 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.437633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.438299 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.438300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.438815 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:58.438816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.439179 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.439180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:58.439245 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.439879 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:58.439885 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:58.441739 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
W20260809 19:30:58.441891 14351 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:58.437195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:43537 (local address 127.31.57.254:41553)
0809 19:30:58.437266 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:58.437268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:58.437286 (+ 18us) server_negotiation.cc:407] Connection header received
0809 19:30:58.437311 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:58.437312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:58.437324 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:58.437394 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:58.437713 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.438211 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.438904 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:58.439097 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:58.439180 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:58.441826 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:43537: BlockingRecv error: recv got EOF from 127.31.57.195:43537 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:30:59.173149 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:59.168737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:34801 (local address 127.31.57.254:41553)
0809 19:30:59.168858 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:59.168860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:59.168868 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:59.168897 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:59.168897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:59.168907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:59.168962 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:59.169278 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.169783 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.170440 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.170616 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.170663 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.173091 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:34801: BlockingRecv error: recv got EOF from 127.31.57.193:34801 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260809 19:30:59.173153 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:59.168509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:34801)
0809 19:30:59.168685 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:59.168698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:59.168775 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:59.169005 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:59.169006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:59.169157 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:59.169208 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.169210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.169852 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.169853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.170368 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.170370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.170679 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.170679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.170723 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.171134 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:59.171138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:59.173028 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":53,"threads_started":1}
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:59.302867 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:59.297939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:54871)
0809 19:30:59.298067 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:59.298078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:59.298133 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:59.298303 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:59.298303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:59.298473 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:59.298524 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.298525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.299490 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.299492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.300018 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.300020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.300517 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.300518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.300579 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.301058 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:59.301066 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:59.302759 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
W20260809 19:30:59.302901 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:59.298043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:54871 (local address 127.31.57.254:41553)
0809 19:30:59.298110 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:59.298112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:59.298126 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:30:59.298200 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:59.298200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:59.298211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:59.298289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:59.298607 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.299336 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.300117 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.300435 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.300504 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.302833 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:54871: BlockingRecv error: recv got EOF from 127.31.57.194:54871 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:30:59 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:30:59.446740 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:59.442280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:54339)
0809 19:30:59.442444 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:59.442454 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:59.442532 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:59.442675 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:59.442676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:59.442851 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:59.442896 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.442898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.443580 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.443581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.444119 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:59.444121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.444487 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.444488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:59.444545 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.444961 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:59.444965 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:59.446652 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
W20260809 19:30:59.446789 14356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:59.442358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54339 (local address 127.31.57.254:41553)
0809 19:30:59.442425 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:59.442427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:59.442518 (+ 91us) server_negotiation.cc:407] Connection header received
0809 19:30:59.442585 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:59.442586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:59.442600 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:59.442668 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:59.442976 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.443504 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.444199 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:59.444414 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:59.444489 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:59.446727 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:54339: BlockingRecv error: recv got EOF from 127.31.57.195:54339 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:00.178222 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:00.173675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:59201)
0809 19:31:00.173825 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:00.173836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:00.173891 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:00.174088 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:00.174088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:00.174252 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:00.174334 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.174336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.174938 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.174939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.175462 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.175463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.175840 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.175841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.175882 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.176384 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:00.176388 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:00.178169 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":55,"threads_started":1}
W20260809 19:31:00.178349 14360 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:00.173800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59201 (local address 127.31.57.254:41553)
0809 19:31:00.173936 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:00.173938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:00.173948 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:31:00.173974 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:00.173975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:00.173988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:00.174074 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:00.174400 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.174876 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.175547 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.175777 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.175828 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.178253 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:59201: BlockingRecv error: recv got EOF from 127.31.57.193:59201 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:00.313202 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:00.303656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:39479)
0809 19:31:00.303822 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:00.303838 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:00.303917 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:00.304050 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:00.304050 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:00.304248 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:00.304304 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.304306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.305007 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.305008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.305550 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.305551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.305955 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.305956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.306022 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.310942 (+ 4920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:00.310948 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:00.313147 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
W20260809 19:31:00.313326 14360 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:00.303704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:39479 (local address 127.31.57.254:41553)
0809 19:31:00.303751 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:00.303754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:00.303901 (+ 147us) server_negotiation.cc:407] Connection header received
0809 19:31:00.303974 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:00.303975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:00.303988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:00.304045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:00.304410 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.304913 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.305640 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.305842 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.305934 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.313245 (+ 7311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:39479: BlockingRecv error: recv got EOF from 127.31.57.194:39479 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:00 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:00.451704 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:00.447240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:55353)
0809 19:31:00.447489 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:00.447499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:00.447556 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:00.447702 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:00.447702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:00.447866 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:00.447916 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.447918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.448522 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.448522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.449006 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:00.449008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.449302 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.449303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:00.449341 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.449750 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:00.449754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:00.451610 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":155,"threads_started":1}
W20260809 19:31:00.451736 14360 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:00.447315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:55353 (local address 127.31.57.254:41553)
0809 19:31:00.447430 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:00.447432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:00.447561 (+ 129us) server_negotiation.cc:407] Connection header received
0809 19:31:00.447622 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:00.447622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:00.447638 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:00.447682 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:00.447981 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.448459 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.449074 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:00.449239 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:00.449284 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:00.451680 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:55353: BlockingRecv error: recv got EOF from 127.31.57.195:55353 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:01.183060 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:01.178649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:56687)
0809 19:31:01.178804 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:01.178817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:01.178881 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:01.179019 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:01.179020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:01.179182 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:01.179234 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.179236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.179872 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.179873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.180380 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.180382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.180695 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.180696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.180736 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.181178 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:01.181182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:01.182990 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
W20260809 19:31:01.183164 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:01.178716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:56687 (local address 127.31.57.254:41553)
0809 19:31:01.178864 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:01.178866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:01.178875 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:31:01.178942 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:01.178943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:01.178952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:01.178999 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:01.179304 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.179806 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.180452 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.180633 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.180683 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.183090 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:56687: BlockingRecv error: recv got EOF from 127.31.57.193:56687 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:01.318423 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:01.313647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:40141)
0809 19:31:01.313779 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:01.313791 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:01.313852 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:01.313997 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:01.313998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:01.314166 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:01.314220 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.314221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.314818 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.314818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.315296 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.315298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.315703 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.315704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.315746 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.316373 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:01.316379 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:01.318362 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
W20260809 19:31:01.318518 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:01.313701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:40141 (local address 127.31.57.254:41553)
0809 19:31:01.313775 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:01.313777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:01.313855 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:31:01.313910 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:01.313911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:01.313923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:01.313979 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:01.314288 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.314754 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.315411 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.315629 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.315703 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.318453 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:40141: BlockingRecv error: recv got EOF from 127.31.57.194:40141 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:01 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:01.456920 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:01.452171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:59455)
0809 19:31:01.452392 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:01.452410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:01.452492 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:01.452655 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:01.452656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:01.452842 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:01.452892 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.452893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.453558 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.453559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.454063 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:01.454064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.454416 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.454417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:01.454472 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.454991 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:01.454995 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:01.456838 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260809 19:31:01.457038 14370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:01.452295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:59455 (local address 127.31.57.254:41553)
0809 19:31:01.452420 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:01.452423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:01.452469 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:31:01.452548 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:01.452549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:01.452564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:01.452641 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:01.452974 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.453464 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.454155 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:01.454331 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:01.454416 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:01.456947 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:59455: BlockingRecv error: recv got EOF from 127.31.57.195:59455 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:02.187839 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:02.183599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:52663)
0809 19:31:02.183745 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:02.183755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:02.183808 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:02.183957 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:02.183957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:02.184141 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:02.184188 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.184190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.184767 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.184768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.185242 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.185243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.185552 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.185553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.185591 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.186076 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:02.186080 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:02.187777 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
W20260809 19:31:02.187937 14376 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:02.183654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:52663 (local address 127.31.57.254:41553)
0809 19:31:02.183786 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:02.183787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:02.183796 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:31:02.183865 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:02.183866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:02.183876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:02.183936 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:02.184243 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.184706 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.185318 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.185491 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.185536 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.187869 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:52663: BlockingRecv error: recv got EOF from 127.31.57.193:52663 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:02.324051 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:02.318916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:44911)
0809 19:31:02.319039 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:02.319050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:02.319105 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:02.319258 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:02.319259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:02.319539 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:02.319619 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.319622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.320329 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.320331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.320889 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.320891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.321238 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.321239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.321322 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.321813 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:02.321819 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:02.323949 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
W20260809 19:31:02.324088 14376 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:02.319017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:44911 (local address 127.31.57.254:41553)
0809 19:31:02.319086 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:02.319088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:02.319098 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:31:02.319155 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:02.319155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:02.319164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:02.319221 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:02.319711 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.320233 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.320978 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.321164 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.321238 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.324028 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:44911: BlockingRecv error: recv got EOF from 127.31.57.194:44911 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:02 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:02.462420 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:02.457463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:53979)
0809 19:31:02.457638 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:02.457650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:02.457740 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:02.457920 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:02.457921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:02.458099 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:02.458144 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.458146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.458828 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.458829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.459347 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:02.459349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.459872 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.459874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:02.459951 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.460518 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:02.460522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:02.462350 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
W20260809 19:31:02.462539 14376 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:02.457548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:53979 (local address 127.31.57.254:41553)
0809 19:31:02.457621 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:02.457624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:02.457722 (+ 98us) server_negotiation.cc:407] Connection header received
0809 19:31:02.457815 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:02.457816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:02.457832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:02.457893 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:02.458227 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.458734 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.459474 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:02.459757 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:02.459844 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:02.462433 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:53979: BlockingRecv error: recv got EOF from 127.31.57.195:53979 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:03.193076 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:03.188344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:42877)
0809 19:31:03.188503 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:03.188514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:03.188580 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:03.188782 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:03.188783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:03.188979 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:03.189049 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.189051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.189684 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.189685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.190208 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.190209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.190580 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.190582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.190643 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.191082 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:03.191086 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:03.192987 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260809 19:31:03.193135 14380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:03.188393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:42877 (local address 127.31.57.254:41553)
0809 19:31:03.188523 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:03.188524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:03.188578 (+ 54us) server_negotiation.cc:407] Connection header received
0809 19:31:03.188662 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:03.188662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:03.188673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:03.188759 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:03.189109 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.189612 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.190309 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.190490 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.190539 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.193060 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:42877: BlockingRecv error: recv got EOF from 127.31.57.193:42877 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:03.329135 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:03.324499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:39967)
0809 19:31:03.324689 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:03.324700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:03.324790 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:03.324953 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:03.324954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:03.325136 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:03.325182 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.325184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.325861 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.325862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.326378 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.326380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.326740 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.326741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.326800 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.327293 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:03.327298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:03.329042 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
W20260809 19:31:03.329190 14380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:03.324590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:39967 (local address 127.31.57.254:41553)
0809 19:31:03.324667 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:03.324670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:03.324772 (+ 102us) server_negotiation.cc:407] Connection header received
0809 19:31:03.324850 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:03.324851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:03.324864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:03.324930 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:03.325275 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.325767 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.326466 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.326655 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.326739 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.329129 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:39967: BlockingRecv error: recv got EOF from 127.31.57.194:39967 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:03 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:03.467703 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:03.462927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:42037)
0809 19:31:03.463099 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:03.463110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:03.463192 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:03.463360 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:03.463361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:03.463591 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:03.463638 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.463640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.464299 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.464300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.464815 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:03.464816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.465166 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.465167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:03.465225 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.465738 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:03.465743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:03.467615 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
W20260809 19:31:03.467741 14380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:03.463009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:42037 (local address 127.31.57.254:41553)
0809 19:31:03.463079 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:03.463081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:03.463174 (+ 93us) server_negotiation.cc:407] Connection header received
0809 19:31:03.463263 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:03.463264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:03.463280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:03.463351 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:03.463721 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.464209 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.464905 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:03.465093 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:03.465168 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:03.467683 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:42037: BlockingRecv error: recv got EOF from 127.31.57.195:42037 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:04.198004 14387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:04.193631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:56909)
0809 19:31:04.193777 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:04.193786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:04.193835 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:04.193970 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:04.193970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:04.194124 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:04.194170 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.194171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.194750 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.194750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.195254 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.195255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.195584 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.195585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.195627 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.196169 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:04.196175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:04.197939 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260809 19:31:04.198123 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:04.193711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:56909 (local address 127.31.57.254:41553)
0809 19:31:04.193842 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:04.193848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:04.193874 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:31:04.193895 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:04.193895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:04.193905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:04.193954 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:04.194231 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.194687 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.195324 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.195522 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.195574 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.198023 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:56909: BlockingRecv error: recv got EOF from 127.31.57.193:56909 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:04.334183 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:04.329607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:49929)
0809 19:31:04.329772 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:04.329783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:04.329867 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:04.330039 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:04.330041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:04.330235 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:04.330285 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.330287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.330969 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.330970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.331519 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.331521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.331860 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.331860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.331914 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.332327 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:04.332331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:04.334122 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260809 19:31:04.334298 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:04.329771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49929 (local address 127.31.57.254:41553)
0809 19:31:04.329849 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:04.329851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:04.329861 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:31:04.329943 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:04.329944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:04.329963 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:04.330030 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:04.330374 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.330883 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.331609 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.331787 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.331861 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.334219 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:49929: BlockingRecv error: recv got EOF from 127.31.57.194:49929 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:04 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:04.472460 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:04.468124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:48271)
0809 19:31:04.468286 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:04.468299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:04.468365 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:04.468498 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:04.468499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:04.468692 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:04.468739 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.468741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.469374 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.469374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.469890 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:04.469891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.470195 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.470196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:04.470233 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.470658 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:04.470663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:04.472403 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
W20260809 19:31:04.472558 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:04.468310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:48271 (local address 127.31.57.254:41553)
0809 19:31:04.468372 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:04.468374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:04.468384 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:31:04.468416 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:04.468417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:04.468427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:04.468475 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:04.468809 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.469304 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.469959 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:04.470133 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:04.470182 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:04.472491 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:48271: BlockingRecv error: recv got EOF from 127.31.57.195:48271 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:05.202991 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:05.198447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:41261)
0809 19:31:05.198615 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:05.198627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:05.198688 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:05.198860 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:05.198861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:05.199026 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:05.199077 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.199078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.199759 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.199760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.200335 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.200337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.200665 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.200666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.200705 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.201149 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:05.201153 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:05.202898 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
W20260809 19:31:05.203043 14397 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:05.198510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:41261 (local address 127.31.57.254:41553)
0809 19:31:05.198642 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:05.198643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:05.198675 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:31:05.198765 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:05.198766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:05.198782 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:05.198840 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:05.199141 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.199656 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.200408 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.200600 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.200652 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.202973 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:41261: BlockingRecv error: recv got EOF from 127.31.57.193:41261 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:05.343483 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:05.334752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:59187)
0809 19:31:05.334936 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:05.334954 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:05.335009 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:05.335160 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:05.335160 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:05.335341 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:05.335447 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.335450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.336108 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.336108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.336623 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.336625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.336923 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.336924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.336965 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.341578 (+ 4613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:05.341583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:05.343361 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
W20260809 19:31:05.343598 14397 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:05.334863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:59187 (local address 127.31.57.254:41553)
0809 19:31:05.334965 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:05.334969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:05.334994 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:31:05.335071 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:05.335072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:05.335083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:05.335140 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:05.335524 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.336029 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.336691 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.336861 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.336910 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.343503 (+ 6593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:59187: BlockingRecv error: recv got EOF from 127.31.57.194:59187 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:05 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.195@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:05.477653 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:05.473142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.195:46987)
0809 19:31:05.473309 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:05.473319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:05.473373 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:05.473509 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:05.473510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:05.473692 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:05.473739 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.473740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.474360 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.474360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.474823 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:05.474825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.475146 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.475147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:05.475183 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.475635 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:05.475639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:05.477596 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
W20260809 19:31:05.477777 14397 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:05.473265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:46987 (local address 127.31.57.254:41553)
0809 19:31:05.473311 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:05.473313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:05.473361 (+ 48us) server_negotiation.cc:407] Connection header received
0809 19:31:05.473430 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:05.473431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:05.473444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:05.473489 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:05.473810 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.474292 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.474911 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:05.475076 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:05.475137 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:05.477678 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.195:46987: BlockingRecv error: recv got EOF from 127.31.57.195:46987 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 19:31:06 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:06 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.193@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:06.207947 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.203716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.193:59405)
0809 19:31:06.203847 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:06.203858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:06.203924 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:06.204049 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:06.204050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:06.204211 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:06.204259 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.204260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.204906 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.204906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.205413 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.205415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.205735 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.205736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.205772 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.206217 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:06.206221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:06.207863 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
W20260809 19:31:06.208030 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:06.203796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59405 (local address 127.31.57.254:41553)
0809 19:31:06.203926 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.203927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.203951 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:31:06.203973 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.203973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.203983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.204030 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.204318 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.204843 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.205483 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.205678 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.205723 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.207951 (+ 2228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.193:59405: BlockingRecv error: recv got EOF from 127.31.57.193:59405 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
Aug 09 19:31:06 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:31:06 dist-test-slave-x4qh krb5kdc[12686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.57.194@KRBTEST.COM for oryx/127.31.57.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:31:06.348511 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.344082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41553 (local address 127.31.57.194:42755)
0809 19:31:06.344222 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:06.344234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:06.344303 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:06.344433 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:06.344434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:06.344592 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:06.344642 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.344644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.345312 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.345313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.345775 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.345784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.346102 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.346103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.346142 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.346624 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:06.346637 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:06.348425 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.57.254:41553: Server oryx/127.31.57.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
W20260809 19:31:06.348547 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:06.344162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42755 (local address 127.31.57.254:41553)
0809 19:31:06.344240 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.344242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.344304 (+ 62us) server_negotiation.cc:407] Connection header received
0809 19:31:06.344358 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.344359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.344369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.344415 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.344699 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.345242 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.345849 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.346039 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.346088 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.348490 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.57.194:42755: BlockingRecv error: recv got EOF from 127.31.57.194:42755 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:31:06.402311 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 13597
I20260809 19:31:06.416131 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 13857
I20260809 19:31:06.425680 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 14080
I20260809 19:31:06.436228 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 13482
2026-08-09T19:31:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17528 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T19:31:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:31:06Z Disabled control of system clock
I20260809 19:31:06.466070 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:44503
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:31:06.544333 14410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:06.544533 14410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:06.544552 14410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:06.545588 14410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:31:06.545632 14410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:06.545644 14410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:06.545678 14410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:31:06.545691 14410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:31:06.546904 14410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:44503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:44503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:06.547081 14410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:06.547293 14410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:06.552191 14422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.552219 14410 server_base.cc:1034] running on GCE node
W20260809 19:31:06.552237 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:06.552191 14425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.552695 14410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:06.553000 14410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:06.554172 14410 hybrid_clock.cc:648] HybridClock initialized: now 1786303866554158 us; error 42 us; skew 500 ppm
I20260809 19:31:06.555560 14410 webserver.cc:460] Webserver started at http://127.31.57.254:39225/ using document root <none> and password file <none>
I20260809 19:31:06.555812 14410 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:06.555883 14410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:06.556033 14410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:06.557003 14410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "10e6001d683e4098b953c377f9178c30"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "8d36577446b87fbb75d8b4cc5a8e015b"
server_key_iv: "74978edd11948a8e947d7970aaf56b21"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.557369 14410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "10e6001d683e4098b953c377f9178c30"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "8d36577446b87fbb75d8b4cc5a8e015b"
server_key_iv: "74978edd11948a8e947d7970aaf56b21"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.558605 14410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:31:06.559310 14440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.559479 14410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:31:06.559587 14410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "10e6001d683e4098b953c377f9178c30"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "8d36577446b87fbb75d8b4cc5a8e015b"
server_key_iv: "74978edd11948a8e947d7970aaf56b21"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.559671 14410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:06.574827 14410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:06.575098 14410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:06.575254 14410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:06.579559 14410 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:44503
I20260809 19:31:06.579591 14511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:44503 every 8 connection(s)
I20260809 19:31:06.579922 14410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:31:06.580557 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 14410
I20260809 19:31:06.580653 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:31:06.580700 14513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:06.580870 31975 external_mini_cluster.cc:1428] Setting key a71c7d5e6c9255915ff29ee670a42b71
I20260809 19:31:06.584128 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30: Bootstrap starting.
I20260809 19:31:06.584743 14513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:06.585049 14513 log.cc:824] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:06.585805 14513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30: No bootstrap required, opened a new log
I20260809 19:31:06.586295 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.581406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.31.57.254:44503)
0809 19:31:06.581602 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.581608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.581632 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:31:06.582082 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.582088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.582138 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.582237 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.582444 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.583284 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.584382 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.584769 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.584915 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.585603 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:06.585606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:06.585606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:06.585748 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:06.585781 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:06.585798 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:06.585846 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:06.586015 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:31:06.586112 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 19:31:06.587988 14513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } }
I20260809 19:31:06.588120 14513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:06.588143 14513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e6001d683e4098b953c377f9178c30, State: Initialized, Role: FOLLOWER
I20260809 19:31:06.588295 14513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } }
I20260809 19:31:06.588370 14513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:31:06.588414 14513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:31:06.588465 14513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:06.589036 14513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } }
I20260809 19:31:06.589145 14513 leader_election.cc:302] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10e6001d683e4098b953c377f9178c30; no voters:
I20260809 19:31:06.589327 14513 leader_election.cc:288] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:31:06.589361 14520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:06.589534 14520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [term 1 LEADER]: Becoming Leader. State: Replica: 10e6001d683e4098b953c377f9178c30, State: Running, Role: LEADER
I20260809 19:31:06.589594 14513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:31:06.589689 14520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } }
I20260809 19:31:06.590057 14522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10e6001d683e4098b953c377f9178c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } } }
I20260809 19:31:06.590139 14522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:06.590312 14525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10e6001d683e4098b953c377f9178c30. Latest consensus state: current_term: 1 leader_uuid: "10e6001d683e4098b953c377f9178c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e6001d683e4098b953c377f9178c30" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 44503 } } }
I20260809 19:31:06.590396 14525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:06.590739 14536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:31:06.591310 14536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:31:06.592692 14536 catalog_manager.cc:1347] Generated new cluster ID: 585b48092ef640bb97267462ed3aa02d
I20260809 19:31:06.592808 14536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:31:06.606827 14536 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:31:06.607509 14536 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:31:06.615239 14536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10e6001d683e4098b953c377f9178c30: Generated new TSK 0
I20260809 19:31:06.615410 14536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:31:06.621004 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--builtin_ntp_servers=127.31.57.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 19:31:06.701735 14552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:06.701948 14552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:06.701982 14552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:06.703022 14552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:06.703078 14552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:06.703115 14552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:31:06.704470 14552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:06.704716 14552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:06.704989 14552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:06.709926 14560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.710006 14552 server_base.cc:1034] running on GCE node
W20260809 19:31:06.709921 14562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:06.710022 14558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.710367 14552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:06.710605 14552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:06.711763 14552 hybrid_clock.cc:648] HybridClock initialized: now 1786303866711740 us; error 56 us; skew 500 ppm
I20260809 19:31:06.712955 14552 webserver.cc:460] Webserver started at http://127.31.57.193:44685/ using document root <none> and password file <none>
I20260809 19:31:06.713184 14552 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:06.713248 14552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:06.713402 14552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:06.714344 14552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "ed8c552b9aa54841ab6fd6a44a5f65b0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "f97f84fc4ff7d014b7aab55df43efcad"
server_key_iv: "3dd08ae8fe2fbe95d0e6a0052b69be08"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.714674 14552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "ed8c552b9aa54841ab6fd6a44a5f65b0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "f97f84fc4ff7d014b7aab55df43efcad"
server_key_iv: "3dd08ae8fe2fbe95d0e6a0052b69be08"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.715948 14552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:31:06.716733 14570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.716907 14552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:31:06.717003 14552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "ed8c552b9aa54841ab6fd6a44a5f65b0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "f97f84fc4ff7d014b7aab55df43efcad"
server_key_iv: "3dd08ae8fe2fbe95d0e6a0052b69be08"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.717082 14552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:06.724997 14552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:06.725251 14552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:06.725396 14552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:06.725608 14552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:06.725903 14552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:06.725963 14552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.726001 14552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:06.726029 14552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.732071 14552 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:37895
I20260809 19:31:06.732121 14758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:37895 every 8 connection(s)
I20260809 19:31:06.732411 14552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:31:06.736572 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 14552
I20260809 19:31:06.736660 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:31:06.736846 31975 external_mini_cluster.cc:1428] Setting key d355aed665ddfa3e9d809f77de14d687
I20260809 19:31:06.737118 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.733012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:58527 (local address 127.31.57.254:44503)
0809 19:31:06.733054 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.733056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.733247 (+ 191us) server_negotiation.cc:407] Connection header received
0809 19:31:06.733671 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.733672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.733683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.733733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.734306 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.734767 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.735809 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.735993 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.736048 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.736760 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:06.736762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:06.736762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:06.736766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:06.736781 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:06.736785 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:06.736830 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:06.736961 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 19:31:06.737018 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:31:06.737201 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.732973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44503 (local address 127.31.57.193:58527)
0809 19:31:06.733138 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:06.733171 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:06.733595 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:06.733785 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:06.733789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:06.733797 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:06.734239 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.734242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.734852 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.734853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.735731 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.735733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.736067 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.736068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.736181 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.736608 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:06.736609 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:06.736646 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:06.736673 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:06.736833 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:06.736837 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:06.736839 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:31:06.736907 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:31:06.737009 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260809 19:31:06.737816 14759 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44503
I20260809 19:31:06.737897 14759 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:06.737887 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--builtin_ntp_servers=127.31.57.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 19:31:06.738078 14759 heartbeater.cc:511] Master 127.31.57.254:44503 requested a full tablet report, sending...
I20260809 19:31:06.738555 14462 ts_manager.cc:194] Registered new tserver with Master: ed8c552b9aa54841ab6fd6a44a5f65b0 (127.31.57.193:37895)
I20260809 19:31:06.739131 14462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.193:58527
W20260809 19:31:06.814769 14767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:06.814944 14767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:06.814961 14767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:06.816045 14767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:06.816087 14767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:06.816136 14767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:31:06.817400 14767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:06.817617 14767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:06.817889 14767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:06.822698 14782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:06.822695 14780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:06.822674 14779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.822955 14767 server_base.cc:1034] running on GCE node
I20260809 19:31:06.823165 14767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:06.823458 14767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:06.824601 14767 hybrid_clock.cc:648] HybridClock initialized: now 1786303866824585 us; error 32 us; skew 500 ppm
I20260809 19:31:06.825740 14767 webserver.cc:460] Webserver started at http://127.31.57.194:40621/ using document root <none> and password file <none>
I20260809 19:31:06.825982 14767 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:06.826045 14767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:06.826170 14767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:06.827117 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "c9d2b4a352214f8dbee02dec162cb562"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "32756ca6c87a7bd4a3afc29e1812e06f"
server_key_iv: "1c7a1472b3f879dba95ff065f2c55d32"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.827484 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "c9d2b4a352214f8dbee02dec162cb562"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "32756ca6c87a7bd4a3afc29e1812e06f"
server_key_iv: "1c7a1472b3f879dba95ff065f2c55d32"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.828677 14767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:31:06.829321 14793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.829478 14767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.829576 14767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "c9d2b4a352214f8dbee02dec162cb562"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "32756ca6c87a7bd4a3afc29e1812e06f"
server_key_iv: "1c7a1472b3f879dba95ff065f2c55d32"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.829654 14767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:06.850411 14767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:06.850670 14767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:06.850812 14767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:06.851024 14767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:06.851328 14767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:06.851406 14767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.851446 14767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:06.851475 14767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.857082 14767 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:45061
I20260809 19:31:06.857134 14983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:45061 every 8 connection(s)
I20260809 19:31:06.857412 14767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:31:06.861756 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.858057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42293 (local address 127.31.57.254:44503)
0809 19:31:06.858122 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.858124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.858328 (+ 204us) server_negotiation.cc:407] Connection header received
0809 19:31:06.858717 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.858718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.858727 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.858773 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.859324 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.859788 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.860628 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.860794 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.860839 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.861546 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:06.861547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:06.861547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:06.861552 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:06.861566 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:06.861569 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:06.861607 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:06.861658 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:31:06.861677 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:31:06.861935 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.858036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44503 (local address 127.31.57.194:42293)
0809 19:31:06.858221 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:06.858251 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:06.858647 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:06.858822 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:06.858824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:06.858831 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:06.859258 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.859261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.859861 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.859862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.860559 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.860560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.860860 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.860860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.860972 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.861426 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:06.861427 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:06.861451 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:06.861471 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:06.861614 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:06.861616 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:06.861618 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:31:06.861659 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:31:06.861726 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260809 19:31:06.862442 14984 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44503
I20260809 19:31:06.862515 14984 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:06.862712 14984 heartbeater.cc:511] Master 127.31.57.254:44503 requested a full tablet report, sending...
I20260809 19:31:06.863107 14462 ts_manager.cc:194] Registered new tserver with Master: c9d2b4a352214f8dbee02dec162cb562 (127.31.57.194:45061)
I20260809 19:31:06.863256 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 14767
I20260809 19:31:06.863337 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:31:06.863505 31975 external_mini_cluster.cc:1428] Setting key 185f468ce25051fe8985e8b43238ca45
I20260809 19:31:06.863546 14462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.194:42293
I20260809 19:31:06.864391 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--builtin_ntp_servers=127.31.57.212:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 19:31:06.941037 14990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:06.941243 14990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:06.941272 14990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:06.942277 14990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:06.942335 14990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:06.942373 14990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:31:06.943590 14990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:06.943807 14990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:06.944048 14990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:06.948952 15003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.949021 14990 server_base.cc:1034] running on GCE node
W20260809 19:31:06.948967 15001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:06.948966 14999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:06.949455 14990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:06.949715 14990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:06.950857 14990 hybrid_clock.cc:648] HybridClock initialized: now 1786303866950842 us; error 39 us; skew 500 ppm
I20260809 19:31:06.952015 14990 webserver.cc:460] Webserver started at http://127.31.57.195:32899/ using document root <none> and password file <none>
I20260809 19:31:06.952229 14990 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:06.952334 14990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:06.952521 14990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:06.953539 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "f518872f3d7f47a98d34cc1e0ac4a4d0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "17381e8aa93a725a4e446fc18a2c40db"
server_key_iv: "12f65c5e89ad44eef782274fd6ebd595"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.953969 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "f518872f3d7f47a98d34cc1e0ac4a4d0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "17381e8aa93a725a4e446fc18a2c40db"
server_key_iv: "12f65c5e89ad44eef782274fd6ebd595"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.955194 14990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:31:06.955945 15011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.956118 14990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.956213 14990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "f518872f3d7f47a98d34cc1e0ac4a4d0"
format_stamp: "Formatted at 2026-08-09 19:31:06 on dist-test-slave-x4qh"
server_key: "17381e8aa93a725a4e446fc18a2c40db"
server_key_iv: "12f65c5e89ad44eef782274fd6ebd595"
server_key_version: "encryptionkey@0"
I20260809 19:31:06.956305 14990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:06.974994 14990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:06.975244 14990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:06.975415 14990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:06.975622 14990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:06.975934 14990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:06.975992 14990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.976027 14990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:06.976056 14990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:06.981364 14990 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:41459
I20260809 19:31:06.981405 15195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:41459 every 8 connection(s)
I20260809 19:31:06.981685 14990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:31:06.986408 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.982370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:37947 (local address 127.31.57.254:44503)
0809 19:31:06.982423 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.982425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.982642 (+ 217us) server_negotiation.cc:407] Connection header received
0809 19:31:06.983175 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.983176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.983187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.983241 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.983852 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.984362 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.985239 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.985413 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.985460 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.986171 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:06.986172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:06.986172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:06.986177 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:06.986190 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:06.986193 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:06.986239 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:06.986331 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 19:31:06.986351 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:31:06.986610 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:06.982355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:44503 (local address 127.31.57.195:37947)
0809 19:31:06.982535 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:06.982563 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:06.983086 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:06.983296 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:06.983299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:06.983307 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:06.983775 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.983779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.984444 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.984444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.985162 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:06.985164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:06.985482 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:06.985483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:06.985585 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:06.986044 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:06.986045 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:06.986069 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:06.986089 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:06.986251 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:06.986255 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:06.986257 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:31:06.986315 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:31:06.986387 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260809 19:31:06.987115 15197 heartbeater.cc:347] Connected to a master server at 127.31.57.254:44503
I20260809 19:31:06.987215 15197 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:06.987349 15197 heartbeater.cc:511] Master 127.31.57.254:44503 requested a full tablet report, sending...
I20260809 19:31:06.987723 14462 ts_manager.cc:194] Registered new tserver with Master: f518872f3d7f47a98d34cc1e0ac4a4d0 (127.31.57.195:41459)
I20260809 19:31:06.988063 14462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.195:37947
I20260809 19:31:06.989740 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 14990
I20260809 19:31:06.989842 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:31:06.989984 31975 external_mini_cluster.cc:1428] Setting key 3d1234a083105870646e45eba0066af1
I20260809 19:31:06.990891 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:31:06.992623 14516 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:06.992132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.31.57.254:44503)
0809 19:31:06.992203 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:06.992205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:06.992345 (+ 140us) server_negotiation.cc:407] Connection header received
0809 19:31:06.992367 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:06.992368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:06.992375 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:06.992420 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:06.992553 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42404: BlockingRecv error: recv got EOF from 127.0.0.1:42404 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 19:31:06.992664 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 14552
I20260809 19:31:07.003943 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 14767
I20260809 19:31:07.018419 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 14990
I20260809 19:31:07.029556 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 14410
2026-08-09T19:31:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (595 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T19:31:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:31:07Z Disabled control of system clock
I20260809 19:31:07.062039 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43067
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:43067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:31:07.138439 15224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:07.138607 15224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:07.138626 15224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:07.139659 15224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:31:07.139694 15224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:07.139707 15224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:07.139719 15224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:31:07.139731 15224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:31:07.140938 15224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:43067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:07.141114 15224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:07.141335 15224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:07.145807 15241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:07.145744 15237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.145977 15224 server_base.cc:1034] running on GCE node
W20260809 19:31:07.145759 15236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.146374 15224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:07.146648 15224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:07.147812 15224 hybrid_clock.cc:648] HybridClock initialized: now 1786303867147797 us; error 28 us; skew 500 ppm
I20260809 19:31:07.148959 15224 webserver.cc:460] Webserver started at http://127.31.57.254:35455/ using document root <none> and password file <none>
I20260809 19:31:07.149199 15224 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:07.149266 15224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:07.149416 15224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:07.150396 15224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "1312952f0efa4d9cba41813ef0d51693"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "e755c54f70e3308be7314529b85a3027"
server_key_iv: "c9aedd67b334258004f6ac29b9f272f6"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.150779 15224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "1312952f0efa4d9cba41813ef0d51693"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "e755c54f70e3308be7314529b85a3027"
server_key_iv: "c9aedd67b334258004f6ac29b9f272f6"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.152082 15224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:31:07.152761 15257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.152949 15224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:31:07.153053 15224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "1312952f0efa4d9cba41813ef0d51693"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "e755c54f70e3308be7314529b85a3027"
server_key_iv: "c9aedd67b334258004f6ac29b9f272f6"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.153124 15224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:07.156384 15224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:07.156621 15224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:07.156757 15224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:07.160661 15224 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:43067
I20260809 19:31:07.160718 15347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:43067 every 8 connection(s)
I20260809 19:31:07.160998 15224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:31:07.161729 15350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:07.164412 15350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693: Bootstrap starting.
I20260809 19:31:07.164927 15350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:07.165158 15350 log.cc:824] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:07.165727 15350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693: No bootstrap required, opened a new log
I20260809 19:31:07.166085 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 15224
I20260809 19:31:07.166183 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:31:07.166363 31975 external_mini_cluster.cc:1428] Setting key cd7fef655ac91aa1cd1b6f0392701a0d
I20260809 19:31:07.167717 15350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } }
I20260809 19:31:07.167888 15350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:07.167928 15350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1312952f0efa4d9cba41813ef0d51693, State: Initialized, Role: FOLLOWER
I20260809 19:31:07.168362 15350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } }
I20260809 19:31:07.168470 15350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:31:07.168514 15350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:31:07.168560 15350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:07.168879 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.166840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.31.57.254:43067)
0809 19:31:07.167027 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:07.167033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:07.167059 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:31:07.167660 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:07.167667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:07.167700 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:07.167805 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:07.168410 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:07.168413 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:07.168413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:07.168416 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:07.168447 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:07.168476 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:07.168558 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:07.168714 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:31:07.168756 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260809 19:31:07.169528 15350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } }
I20260809 19:31:07.169634 15350 leader_election.cc:302] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1312952f0efa4d9cba41813ef0d51693; no voters:
I20260809 19:31:07.169787 15350 leader_election.cc:288] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:31:07.169848 15355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:07.170007 15355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [term 1 LEADER]: Becoming Leader. State: Replica: 1312952f0efa4d9cba41813ef0d51693, State: Running, Role: LEADER
I20260809 19:31:07.170027 15350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:31:07.170094 15355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } }
I20260809 19:31:07.170454 15359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1312952f0efa4d9cba41813ef0d51693. Latest consensus state: current_term: 1 leader_uuid: "1312952f0efa4d9cba41813ef0d51693" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } } }
I20260809 19:31:07.170461 15358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1312952f0efa4d9cba41813ef0d51693" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1312952f0efa4d9cba41813ef0d51693" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43067 } } }
I20260809 19:31:07.170581 15359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:07.170610 15358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:07.170789 15374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:31:07.171309 15374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:31:07.172919 15374 catalog_manager.cc:1347] Generated new cluster ID: d8de512763524eb29621139d91442d84
I20260809 19:31:07.172972 15374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:31:07.179790 15374 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:31:07.179947 15374 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:31:07.183419 15374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1312952f0efa4d9cba41813ef0d51693: Generated new TSK 0
I20260809 19:31:07.183575 15374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:31:07.185688 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--builtin_ntp_servers=127.31.57.212:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:31:07.265619 15391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:07.265795 15391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:07.265815 15391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:07.266960 15391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:07.266997 15391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:07.267021 15391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:31:07.268347 15391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:07.268577 15391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:07.268878 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:07.274150 15400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:07.274262 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:07.274125 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.274578 15391 server_base.cc:1034] running on GCE node
I20260809 19:31:07.274801 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:07.275064 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:07.276209 15391 hybrid_clock.cc:648] HybridClock initialized: now 1786303867276191 us; error 34 us; skew 500 ppm
I20260809 19:31:07.277490 15391 webserver.cc:460] Webserver started at http://127.31.57.193:37813/ using document root <none> and password file <none>
I20260809 19:31:07.277738 15391 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:07.277817 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:07.277976 15391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:07.278959 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "39bb43a232bf4fb8a9fb21034a27390d"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "964a8c3726daed24d047f9c924b8d54d"
server_key_iv: "1b110d46f676022a8c7974ccac2625b3"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.279297 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "39bb43a232bf4fb8a9fb21034a27390d"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "964a8c3726daed24d047f9c924b8d54d"
server_key_iv: "1b110d46f676022a8c7974ccac2625b3"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.280591 15391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:31:07.281342 15412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.281548 15391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:31:07.281636 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "39bb43a232bf4fb8a9fb21034a27390d"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "964a8c3726daed24d047f9c924b8d54d"
server_key_iv: "1b110d46f676022a8c7974ccac2625b3"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.281703 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:07.286393 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:07.286651 15391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:07.286772 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:07.286957 15391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:07.287261 15391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:07.287297 15391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.287320 15391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:07.287338 15391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.293742 15391 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:44125
I20260809 19:31:07.293771 15590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:44125 every 8 connection(s)
I20260809 19:31:07.294091 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:31:07.294364 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 15391
I20260809 19:31:07.294441 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:31:07.294623 31975 external_mini_cluster.cc:1428] Setting key bc60a61d0cf0c70efa6dd3e30e92ff67
I20260809 19:31:07.295713 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--builtin_ntp_servers=127.31.57.212:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 19:31:07.297353 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.295035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:44971 (local address 127.31.57.254:43067)
0809 19:31:07.295074 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:07.295076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:07.295135 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:31:07.295738 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:07.295740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:07.295746 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:07.295794 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:07.297101 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:07.297102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:07.297103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:07.297107 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:07.297121 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:07.297126 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:07.297174 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:07.297264 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 19:31:07.297284 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:31:07.297643 15595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.294779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43067 (local address 127.31.57.193:44971)
0809 19:31:07.295027 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:07.295065 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:07.295635 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:07.295969 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:07.295972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:07.295982 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:07.296945 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:07.296946 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:07.296972 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:07.296999 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:07.297172 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:07.297177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:07.297180 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:31:07.297273 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 19:31:07.297346 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260809 19:31:07.298273 15592 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43067
I20260809 19:31:07.298373 15592 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:07.298516 15592 heartbeater.cc:511] Master 127.31.57.254:43067 requested a full tablet report, sending...
I20260809 19:31:07.299045 15285 ts_manager.cc:194] Registered new tserver with Master: 39bb43a232bf4fb8a9fb21034a27390d (127.31.57.193:44125)
W20260809 19:31:07.376905 15596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:07.377111 15596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:07.377149 15596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:07.378209 15596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:07.378271 15596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:07.378312 15596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:31:07.379590 15596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:07.379827 15596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:07.380087 15596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:07.385015 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:07.384982 15612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.384999 15596 server_base.cc:1034] running on GCE node
W20260809 19:31:07.384968 15609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.385361 15596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:07.385646 15596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:07.386809 15596 hybrid_clock.cc:648] HybridClock initialized: now 1786303867386803 us; error 37 us; skew 500 ppm
I20260809 19:31:07.389765 15596 webserver.cc:460] Webserver started at http://127.31.57.194:42035/ using document root <none> and password file <none>
I20260809 19:31:07.390013 15596 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:07.390079 15596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:07.390213 15596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:07.391211 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "7256a0f4e1c24919a6bbcb60e497c8f4"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "53c2a7804f7d657ed948efbb08cfa632"
server_key_iv: "d8fc7284f027760fd032f8dd6e9c5d76"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.391587 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "7256a0f4e1c24919a6bbcb60e497c8f4"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "53c2a7804f7d657ed948efbb08cfa632"
server_key_iv: "d8fc7284f027760fd032f8dd6e9c5d76"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.392851 15596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:31:07.393539 15621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.393723 15596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:31:07.393818 15596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "7256a0f4e1c24919a6bbcb60e497c8f4"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "53c2a7804f7d657ed948efbb08cfa632"
server_key_iv: "d8fc7284f027760fd032f8dd6e9c5d76"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.393903 15596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:07.397184 15596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:07.397464 15596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:07.397603 15596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:07.397830 15596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:07.398118 15596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:07.398171 15596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.398202 15596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:07.398227 15596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.403746 15596 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:38703
I20260809 19:31:07.403822 15822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:38703 every 8 connection(s)
I20260809 19:31:07.404111 15596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:31:07.406787 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.404842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:37649 (local address 127.31.57.254:43067)
0809 19:31:07.404888 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:07.404890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:07.405076 (+ 186us) server_negotiation.cc:407] Connection header received
0809 19:31:07.405476 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:07.405477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:07.405481 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:07.405534 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:07.406588 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:07.406590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:07.406590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:07.406594 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:07.406609 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:07.406613 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:07.406658 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:07.406722 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:31:07.406733 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:31:07.406998 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.404798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43067 (local address 127.31.57.194:37649)
0809 19:31:07.404971 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:07.405007 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:07.405398 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:07.405613 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:07.405616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:07.405624 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:07.406451 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:07.406452 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:07.406478 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:07.406504 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:07.406657 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:07.406659 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:07.406662 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:31:07.406730 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:31:07.406774 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260809 19:31:07.407529 15826 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43067
I20260809 19:31:07.407626 15826 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:07.407770 15826 heartbeater.cc:511] Master 127.31.57.254:43067 requested a full tablet report, sending...
I20260809 19:31:07.408149 15285 ts_manager.cc:194] Registered new tserver with Master: 7256a0f4e1c24919a6bbcb60e497c8f4 (127.31.57.194:38703)
I20260809 19:31:07.412015 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 15596
I20260809 19:31:07.412088 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:31:07.412221 31975 external_mini_cluster.cc:1428] Setting key 79e88daa65574f54f362c59122e58c18
I20260809 19:31:07.413056 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--builtin_ntp_servers=127.31.57.212:45753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:31:07.493853 15830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:07.494041 15830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:07.494062 15830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:07.495123 15830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:07.495162 15830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:07.495208 15830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:31:07.496608 15830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:07.496858 15830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:07.497145 15830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:07.502094 15846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:07.502146 15841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.502175 15830 server_base.cc:1034] running on GCE node
W20260809 19:31:07.502102 15842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:07.502509 15830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:07.502756 15830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:07.503888 15830 hybrid_clock.cc:648] HybridClock initialized: now 1786303867503875 us; error 30 us; skew 500 ppm
I20260809 19:31:07.505088 15830 webserver.cc:460] Webserver started at http://127.31.57.195:37405/ using document root <none> and password file <none>
I20260809 19:31:07.505354 15830 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:07.505400 15830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:07.505563 15830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:07.506613 15830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "6959c0e14ea14afe8b76b452f467425c"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "d92b5330110f62b35614d40721dfb3b9"
server_key_iv: "19cc10fc211a562fc46a69b63e35af71"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.507030 15830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "6959c0e14ea14afe8b76b452f467425c"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "d92b5330110f62b35614d40721dfb3b9"
server_key_iv: "19cc10fc211a562fc46a69b63e35af71"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.508608 15830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:31:07.509917 15855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.510329 15830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:31:07.510418 15830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "6959c0e14ea14afe8b76b452f467425c"
format_stamp: "Formatted at 2026-08-09 19:31:07 on dist-test-slave-x4qh"
server_key: "d92b5330110f62b35614d40721dfb3b9"
server_key_iv: "19cc10fc211a562fc46a69b63e35af71"
server_key_version: "encryptionkey@0"
I20260809 19:31:07.510488 15830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:07.514364 15830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:07.514557 15830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:07.514647 15830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:07.514794 15830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:07.515017 15830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:07.515044 15830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.515064 15830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:07.515075 15830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:07.520579 15830 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:44325
I20260809 19:31:07.520642 16044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:44325 every 8 connection(s)
I20260809 19:31:07.520891 15830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:31:07.523474 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.521542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:45551 (local address 127.31.57.254:43067)
0809 19:31:07.521595 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:07.521597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:07.521808 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:31:07.522168 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:07.522169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:07.522174 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:07.522222 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:07.523211 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:07.523212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:07.523212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:31:07.523215 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:07.523228 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:31:07.523231 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:07.523275 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:07.523336 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:31:07.523348 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:31:07.523608 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:07.521524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43067 (local address 127.31.57.195:45551)
0809 19:31:07.521701 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:07.521729 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:07.522096 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:07.522264 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:07.522267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:07.522274 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:07.523081 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:31:07.523081 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:07.523104 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:31:07.523130 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:07.523287 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:07.523291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:07.523293 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:31:07.523344 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:31:07.523428 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260809 19:31:07.524044 16045 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43067
I20260809 19:31:07.524140 16045 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:07.524281 16045 heartbeater.cc:511] Master 127.31.57.254:43067 requested a full tablet report, sending...
I20260809 19:31:07.524673 15285 ts_manager.cc:194] Registered new tserver with Master: 6959c0e14ea14afe8b76b452f467425c (127.31.57.195:44325)
I20260809 19:31:07.527998 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 15830
I20260809 19:31:07.528069 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:31:07.528206 31975 external_mini_cluster.cc:1428] Setting key f301791a3b2548997c3efe2d0bf59993
I20260809 19:31:07.529044 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:31:07.530655 15354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:31:07.530247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.31.57.254:43067)
0809 19:31:07.530328 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:07.530329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:07.530375 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:31:07.530418 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:07.530418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:07.530421 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:07.530465 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:07.530596 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43946: BlockingRecv error: recv got EOF from 127.0.0.1:43946 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 19:31:07.530759 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 15391
I20260809 19:31:07.540984 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 15596
I20260809 19:31:07.550736 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 15830
I20260809 19:31:07.560607 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 15224
2026-08-09T19:31:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (532 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:31:07 dist-test-slave-x4qh krb5kdc[16065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:31:07 dist-test-slave-x4qh krb5kdc[16065](info): set up 2 sockets
Aug 09 19:31:07 dist-test-slave-x4qh krb5kdc[16065](info): commencing operation
krb5kdc: starting...
W20260809 19:31:08.785291 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:31:08 dist-test-slave-x4qh krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:31:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:31:08.897338 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:35293
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:08.973814 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:08.973982 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:08.974002 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:08.975006 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:31:08.975039 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:08.975052 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:08.975064 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:31:08.975076 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:31:08.976492 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:35293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:08.976661 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:08.976817 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:08.981244 16103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:08.981251 16104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:08.981251 16106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:08.981514 16094 server_base.cc:1034] running on GCE node
I20260809 19:31:08.981801 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:08.982088 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:08.983251 16094 hybrid_clock.cc:648] HybridClock initialized: now 1786303868983237 us; error 46 us; skew 500 ppm
Aug 09 19:31:08 dist-test-slave-x4qh krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303868, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:08.985635 16094 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:31:08.986025 16094 webserver.cc:460] Webserver started at http://127.31.57.254:41471/ using document root <none> and password file <none>
I20260809 19:31:08.986275 16094 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:08.986346 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:08.986485 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:08.987542 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "37c3ed710aca4b5db92bba03ff6a3d86"
format_stamp: "Formatted at 2026-08-09 19:31:08 on dist-test-slave-x4qh"
server_key: "51a3531ba8e68dda4557cd1da5cd4985"
server_key_iv: "357b539b86672cf46df6cb346ea6b543"
server_key_version: "encryptionkey@0"
I20260809 19:31:08.987951 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "37c3ed710aca4b5db92bba03ff6a3d86"
format_stamp: "Formatted at 2026-08-09 19:31:08 on dist-test-slave-x4qh"
server_key: "51a3531ba8e68dda4557cd1da5cd4985"
server_key_iv: "357b539b86672cf46df6cb346ea6b543"
server_key_version: "encryptionkey@0"
I20260809 19:31:08.989243 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:31:08.989928 16118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:08.990088 16094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:31:08.990195 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "37c3ed710aca4b5db92bba03ff6a3d86"
format_stamp: "Formatted at 2026-08-09 19:31:08 on dist-test-slave-x4qh"
server_key: "51a3531ba8e68dda4557cd1da5cd4985"
server_key_iv: "357b539b86672cf46df6cb346ea6b543"
server_key_version: "encryptionkey@0"
I20260809 19:31:08.990262 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:09.012634 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:09.014219 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:09.014405 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:09.018226 16094 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:35293
I20260809 19:31:09.018275 16213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:35293 every 8 connection(s)
I20260809 19:31:09.018633 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:31:09.019708 16214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:09.021719 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 16094
I20260809 19:31:09.021803 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:31:09.021951 16214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: Bootstrap starting.
I20260809 19:31:09.021994 31975 external_mini_cluster.cc:1428] Setting key 7b89793182cca7f06f7de7378fe763af
I20260809 19:31:09.022675 16214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:09.023005 16214 log.cc:824] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:09.023762 16214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: No bootstrap required, opened a new log
I20260809 19:31:09.025466 16214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } }
I20260809 19:31:09.025610 16214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:09.025657 16214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c3ed710aca4b5db92bba03ff6a3d86, State: Initialized, Role: FOLLOWER
I20260809 19:31:09.025811 16214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } }
I20260809 19:31:09.025916 16214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:31:09.025960 16214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:31:09.026007 16214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:09.026553 16214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } }
I20260809 19:31:09.026664 16214 leader_election.cc:302] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c3ed710aca4b5db92bba03ff6a3d86; no voters:
I20260809 19:31:09.026875 16214 leader_election.cc:288] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:31:09.026916 16224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:09.027071 16224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [term 1 LEADER]: Becoming Leader. State: Replica: 37c3ed710aca4b5db92bba03ff6a3d86, State: Running, Role: LEADER
I20260809 19:31:09.027133 16214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:31:09.027164 16224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } }
I20260809 19:31:09.027511 16226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37c3ed710aca4b5db92bba03ff6a3d86. Latest consensus state: current_term: 1 leader_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } } }
I20260809 19:31:09.027582 16226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:09.027791 16231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:31:09.028244 16231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:09.029791 16231 catalog_manager.cc:1347] Generated new cluster ID: e907b9fe9bf946fabd414cb78c2b4873
I20260809 19:31:09.029892 16231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:31:09.028831 16225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c3ed710aca4b5db92bba03ff6a3d86" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 35293 } } }
I20260809 19:31:09.029986 16225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:09.033277 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.022493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.31.57.254:35293)
0809 19:31:09.022697 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.022706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.022740 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:31:09.023327 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.023335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.023437 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.023593 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:09.024293 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.025119 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.026019 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.026441 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.026520 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.029877 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:09.029881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:09.029882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:09.029893 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:09.031734 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.032116 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.032119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.032119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.032208 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.032449 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.032450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.032450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.032677 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:09.032860 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.032943 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.033080 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260809 19:31:09.039018 16231 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:31:09.039732 16231 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:31:09.046034 16231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: Generated new TSK 0
I20260809 19:31:09.046190 16231 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:31:09.073622 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:09.148159 16254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:09.148324 16254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:09.148345 16254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:09.149353 16254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:09.149389 16254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:09.149435 16254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:31:09.150861 16254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:09.151082 16254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:09.151316 16254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:09.155803 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.155783 16265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.155784 16263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:09.156100 16254 server_base.cc:1034] running on GCE node
I20260809 19:31:09.156353 16254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:09.156582 16254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:09.157734 16254 hybrid_clock.cc:648] HybridClock initialized: now 1786303869157731 us; error 39 us; skew 500 ppm
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:09.159962 16254 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:31:09.160283 16254 webserver.cc:460] Webserver started at http://127.31.57.193:42533/ using document root <none> and password file <none>
I20260809 19:31:09.160444 16254 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:09.160477 16254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:09.160552 16254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:09.161396 16254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "34b60020e3714a9c9818a77a12b1968a"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "93d1fcc072f8fbef4df496880d56c5bd"
server_key_iv: "d314fa646b235fd618a0df405c05ab01"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.161731 16254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "34b60020e3714a9c9818a77a12b1968a"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "93d1fcc072f8fbef4df496880d56c5bd"
server_key_iv: "d314fa646b235fd618a0df405c05ab01"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.162897 16254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:31:09.163650 16281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.163825 16254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:31:09.163911 16254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "34b60020e3714a9c9818a77a12b1968a"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "93d1fcc072f8fbef4df496880d56c5bd"
server_key_iv: "d314fa646b235fd618a0df405c05ab01"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.163961 16254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:09.179291 16254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:09.181286 16254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:09.181384 16254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:09.181548 16254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:09.181780 16254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:09.181811 16254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.181830 16254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:09.181843 16254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.187582 16254 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:39965
I20260809 19:31:09.187655 16478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:39965 every 8 connection(s)
I20260809 19:31:09.187945 16254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:09.196254 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.188739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:41437 (local address 127.31.57.254:35293)
0809 19:31:09.188805 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.188807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.188982 (+ 175us) server_negotiation.cc:407] Connection header received
0809 19:31:09.189404 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.189405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.189412 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.189458 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:09.190266 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.190744 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.191695 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.191877 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.191928 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.193822 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:09.193823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:09.193823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:09.193830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:09.195190 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.195566 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.195566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.195567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.195617 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.195805 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.195806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.195806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.195885 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:09.195925 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.196097 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.196180 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:31:09.196410 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.188691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:35293 (local address 127.31.57.193:41437)
0809 19:31:09.188862 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.188899 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.189331 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.189508 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.189511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.190022 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:09.190191 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.190195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.190826 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.190826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.191626 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.191629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.191944 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.191945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.192031 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.192439 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:09.192446 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:09.193690 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:09.195254 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.195259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.195261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.195515 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.195675 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.195676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.195676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.195770 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.195929 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:09.195933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:09.196020 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.196087 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.196207 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260809 19:31:09.196889 16479 heartbeater.cc:347] Connected to a master server at 127.31.57.254:35293
I20260809 19:31:09.196975 16479 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:09.197096 16479 heartbeater.cc:511] Master 127.31.57.254:35293 requested a full tablet report, sending...
I20260809 19:31:09.197571 16157 ts_manager.cc:194] Registered new tserver with Master: 34b60020e3714a9c9818a77a12b1968a (127.31.57.193:39965)
I20260809 19:31:09.197906 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 16254
I20260809 19:31:09.197999 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:31:09.198168 31975 external_mini_cluster.cc:1428] Setting key b9fbd6ea58d2d1c567debca2277cef97
I20260809 19:31:09.198413 16157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:41437
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:31:09.221817 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:09.296649 16486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:09.296837 16486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:09.296859 16486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:09.298049 16486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:09.298105 16486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:09.298131 16486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:31:09.299517 16486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:09.299678 16486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:09.299889 16486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:09.304458 16493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.304605 16495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.304481 16491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:09.304695 16486 server_base.cc:1034] running on GCE node
I20260809 19:31:09.304900 16486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:09.305135 16486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:09.306301 16486 hybrid_clock.cc:648] HybridClock initialized: now 1786303869306291 us; error 30 us; skew 500 ppm
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:09.308661 16486 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:31:09.309010 16486 webserver.cc:460] Webserver started at http://127.31.57.194:37869/ using document root <none> and password file <none>
I20260809 19:31:09.309163 16486 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:09.309198 16486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:09.309276 16486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:09.310159 16486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "7b31d7a118124fe5babf21c3a30da414"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "ce057f840e4ee961720680397755658e"
server_key_iv: "b3903ab1789afb6f8b6257579e483aa7"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.310436 16486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "7b31d7a118124fe5babf21c3a30da414"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "ce057f840e4ee961720680397755658e"
server_key_iv: "b3903ab1789afb6f8b6257579e483aa7"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.311620 16486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:31:09.312268 16508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.312435 16486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.312529 16486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "7b31d7a118124fe5babf21c3a30da414"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "ce057f840e4ee961720680397755658e"
server_key_iv: "b3903ab1789afb6f8b6257579e483aa7"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.312637 16486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:09.318372 16486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:09.320034 16486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:09.320178 16486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:09.320376 16486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:09.320662 16486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:09.320715 16486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.320751 16486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:09.320778 16486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.326050 16486 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:43335
I20260809 19:31:09.326116 16691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:43335 every 8 connection(s)
I20260809 19:31:09.326411 16486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:09.334558 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.327101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:37519 (local address 127.31.57.254:35293)
0809 19:31:09.327163 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.327165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.327312 (+ 147us) server_negotiation.cc:407] Connection header received
0809 19:31:09.327797 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.327799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.327811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.327854 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:09.328803 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.329319 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.330290 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.330490 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.330567 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.332437 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:09.332438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:09.332438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:09.332444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:09.333507 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.333837 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.333838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.333838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.333888 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.334080 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.334081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.334081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.334170 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:09.334216 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.334421 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.334480 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:31:09.334677 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.327022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:35293 (local address 127.31.57.194:37519)
0809 19:31:09.327196 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.327237 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.327686 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.327927 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.327930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.328528 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:09.328695 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.328699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.329421 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.329422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.330176 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.330178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.330575 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.330576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.330716 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.331095 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:09.331103 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:09.332359 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:09.333573 (+ 1214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.333579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.333580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.333774 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.333945 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.333945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.333945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.334029 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.334221 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:09.334225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:09.334322 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.334392 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.334509 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260809 19:31:09.335180 16694 heartbeater.cc:347] Connected to a master server at 127.31.57.254:35293
I20260809 19:31:09.335282 16694 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:09.335472 16694 heartbeater.cc:511] Master 127.31.57.254:35293 requested a full tablet report, sending...
I20260809 19:31:09.335855 16156 ts_manager.cc:194] Registered new tserver with Master: 7b31d7a118124fe5babf21c3a30da414 (127.31.57.194:43335)
I20260809 19:31:09.336103 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 16486
I20260809 19:31:09.336199 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:31:09.336288 16156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:37519
I20260809 19:31:09.336408 31975 external_mini_cluster.cc:1428] Setting key e42f55ae2464c34b582caa135d7f4fa4
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:31:09.359846 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:09.436196 16703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:09.436367 16703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:09.436388 16703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:09.437392 16703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:09.437436 16703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:09.437459 16703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:31:09.438678 16703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:09.438831 16703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:09.438974 16703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:09.443717 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.443723 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:09.443723 16713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:09.443913 16703 server_base.cc:1034] running on GCE node
I20260809 19:31:09.444089 16703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:09.444295 16703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:09.445477 16703 hybrid_clock.cc:648] HybridClock initialized: now 1786303869445482 us; error 53 us; skew 500 ppm
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:09.447933 16703 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:31:09.448351 16703 webserver.cc:460] Webserver started at http://127.31.57.195:38251/ using document root <none> and password file <none>
I20260809 19:31:09.448527 16703 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:09.448581 16703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:09.448671 16703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:31:09.449577 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "760f2c4df2154a958eb10f08900cbfa2"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "8e71e5a683285e1377ed8c7dabb498e2"
server_key_iv: "d0c39fb9d0b29274499b45c650cc01b0"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.449939 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "760f2c4df2154a958eb10f08900cbfa2"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "8e71e5a683285e1377ed8c7dabb498e2"
server_key_iv: "d0c39fb9d0b29274499b45c650cc01b0"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.451169 16703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:31:09.451928 16733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.452107 16703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.452186 16703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "760f2c4df2154a958eb10f08900cbfa2"
format_stamp: "Formatted at 2026-08-09 19:31:09 on dist-test-slave-x4qh"
server_key: "8e71e5a683285e1377ed8c7dabb498e2"
server_key_iv: "d0c39fb9d0b29274499b45c650cc01b0"
server_key_version: "encryptionkey@0"
I20260809 19:31:09.452235 16703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:09.476999 16703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:09.479015 16703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:09.479130 16703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:09.479290 16703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:09.479525 16703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:09.479554 16703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.479573 16703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:09.479583 16703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:09.484774 16703 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:34167
I20260809 19:31:09.484836 16927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:34167 every 8 connection(s)
I20260809 19:31:09.485103 16703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:09.493244 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.485751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50965 (local address 127.31.57.254:35293)
0809 19:31:09.485819 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.485821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.486001 (+ 180us) server_negotiation.cc:407] Connection header received
0809 19:31:09.486501 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.486502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.486512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.486561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:09.487401 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.487872 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.488704 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.488870 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.488916 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.490665 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:09.490666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:09.490667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:09.490673 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:09.491893 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.492350 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.492351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.492351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.492414 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.492673 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.492674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.492674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.492815 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:09.492868 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.493123 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.493183 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:31:09.493428 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.485723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:35293 (local address 127.31.57.195:50965)
0809 19:31:09.485886 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.485923 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.486412 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.486617 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.486619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.487126 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:09.487295 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.487298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.487946 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.487947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.488631 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.488632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.488937 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.488938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.489028 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.489383 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:09.489390 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:09.490588 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:09.491990 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.491996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.491998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.492252 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.492481 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:09.492481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:09.492482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:09.492583 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:09.492886 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:09.492892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:09.493005 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.493093 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.493207 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260809 19:31:09.493981 16928 heartbeater.cc:347] Connected to a master server at 127.31.57.254:35293
I20260809 19:31:09.494095 16928 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:09.494156 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 16703
I20260809 19:31:09.494256 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:31:09.494290 16928 heartbeater.cc:511] Master 127.31.57.254:35293 requested a full tablet report, sending...
I20260809 19:31:09.494458 31975 external_mini_cluster.cc:1428] Setting key a45bcf8ca90274395dc7a657819eb2c8
I20260809 19:31:09.494799 16156 ts_manager.cc:194] Registered new tserver with Master: 760f2c4df2154a958eb10f08900cbfa2 (127.31.57.195:34167)
I20260809 19:31:09.495256 16156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:50965
I20260809 19:31:09.495383 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:31:09.502418 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.496744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.31.57.254:35293)
0809 19:31:09.496823 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.496825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.496908 (+ 83us) server_negotiation.cc:407] Connection header received
0809 19:31:09.496973 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.496974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.496983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.497037 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:09.497612 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.498067 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.498773 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.498970 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.499044 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.500186 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:09.500187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:09.500187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:09.500194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:09.501312 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.501654 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.501655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.501655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.501709 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:09.501913 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:09.501914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:09.501915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:09.502050 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:09.502169 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.502264 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.502334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 19:31:09.503722 16156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:31:09.505249 16156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:31:09.511175 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.507500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34167 (local address 127.0.0.1:59708)
0809 19:31:09.508044 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.508082 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.508146 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.508439 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.508442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.508444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:09.508505 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.508507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.509373 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.509374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.510196 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.510199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.510877 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.510878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.510964 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.510974 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.511056 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.511109 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":52,"threads_started":1}
I20260809 19:31:09.511564 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.507955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.31.57.195:34167)
0809 19:31:09.508287 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.508291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.508305 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:31:09.508331 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.508332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.508343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.508433 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:09.508587 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.509185 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.510313 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.510763 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.510864 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.510918 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.511075 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.511109 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":68,"threads_started":1}
I20260809 19:31:09.512151 16813 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c674145374270bfc246ca5a9c979a (DEFAULT_TABLE table=test-table [id=50c96df1620c4db99d6b0c3924e3916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:09.512465 16813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c674145374270bfc246ca5a9c979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:09.515113 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.507743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.31.57.194:43335)
0809 19:31:09.507975 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.507980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.511355 (+ 3375us) server_negotiation.cc:407] Connection header received
0809 19:31:09.511424 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.511427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.511443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.511509 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:09.511697 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.512664 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.513896 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.514613 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.514683 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.514732 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.514989 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.515029 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":84,"threads_started":1}
I20260809 19:31:09.515432 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.507500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43335 (local address 127.0.0.1:49072)
0809 19:31:09.511234 (+ 3734us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.511244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.511294 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.511585 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.511586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.511587 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:09.511618 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.511619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.512797 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.512798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.513795 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.513798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.514765 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.514766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.514820 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.514824 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.514870 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.514912 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3726,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260809 19:31:09.516067 16588 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c674145374270bfc246ca5a9c979a (DEFAULT_TABLE table=test-table [id=50c96df1620c4db99d6b0c3924e3916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:09.516454 16588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c674145374270bfc246ca5a9c979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:09.517472 16950 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Bootstrap starting.
I20260809 19:31:09.518144 16950 tablet_bootstrap.cc:654] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:09.518716 16950 log.cc:824] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:09.519614 16950 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: No bootstrap required, opened a new log
I20260809 19:31:09.519693 16950 ts_tablet_manager.cc:1397] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:31:09.520073 16953 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Bootstrap starting.
I20260809 19:31:09.520821 16953 tablet_bootstrap.cc:654] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:09.521473 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.507704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:39965 (local address 127.0.0.1:54694)
0809 19:31:09.515426 (+ 7722us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.515439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.515509 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.516746 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.516747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.516749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:09.516806 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.516807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.518967 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.518968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.520489 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.520492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.521223 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.521224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.521286 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.521291 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.521352 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.521392 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7704,"thread_start_us":64,"threads_started":1}
I20260809 19:31:09.521486 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.508104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.31.57.193:39965)
0809 19:31:09.508551 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.508555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.516191 (+ 7636us) server_negotiation.cc:407] Connection header received
0809 19:31:09.516239 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.516241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.516281 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.516360 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:09.516897 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.518811 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.520586 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.521132 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.521221 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.521287 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.521366 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.521407 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":79,"threads_started":1}
I20260809 19:31:09.521466 16953 log.cc:824] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:09.521941 16950 raft_consensus.cc:348] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.522101 16950 raft_consensus.cc:374] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:09.522135 16950 raft_consensus.cc:729] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760f2c4df2154a958eb10f08900cbfa2, State: Initialized, Role: FOLLOWER
I20260809 19:31:09.522236 16950 consensus_queue.cc:260] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.522293 16365 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7c674145374270bfc246ca5a9c979a (DEFAULT_TABLE table=test-table [id=50c96df1620c4db99d6b0c3924e3916a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:09.522441 16953 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: No bootstrap required, opened a new log
I20260809 19:31:09.522498 16953 ts_tablet_manager.cc:1397] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:31:09.522513 16950 ts_tablet_manager.cc:1428] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:31:09.522614 16928 heartbeater.cc:503] Master 127.31.57.254:35293 was elected leader, sending a full tablet report...
I20260809 19:31:09.522619 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7c674145374270bfc246ca5a9c979a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:09.524434 16953 raft_consensus.cc:348] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.524595 16953 raft_consensus.cc:374] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:09.524639 16953 raft_consensus.cc:729] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b31d7a118124fe5babf21c3a30da414, State: Initialized, Role: FOLLOWER
I20260809 19:31:09.524771 16953 consensus_queue.cc:260] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.525067 16953 ts_tablet_manager.cc:1428] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:31:09.525120 16694 heartbeater.cc:503] Master 127.31.57.254:35293 was elected leader, sending a full tablet report...
I20260809 19:31:09.526793 16962 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: Bootstrap starting.
I20260809 19:31:09.527539 16962 tablet_bootstrap.cc:654] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:09.528061 16962 log.cc:824] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:09.528757 16962 tablet_bootstrap.cc:492] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: No bootstrap required, opened a new log
I20260809 19:31:09.528820 16962 ts_tablet_manager.cc:1397] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:31:09.530201 16962 raft_consensus.cc:348] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.530390 16962 raft_consensus.cc:374] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:09.530444 16962 raft_consensus.cc:729] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b60020e3714a9c9818a77a12b1968a, State: Initialized, Role: FOLLOWER
I20260809 19:31:09.530576 16962 consensus_queue.cc:260] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.530790 16962 ts_tablet_manager.cc:1428] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:31:09.530915 16479 heartbeater.cc:503] Master 127.31.57.254:35293 was elected leader, sending a full tablet report...
W20260809 19:31:09.577416 16695 tablet.cc:2369] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:31:09.635018 16964 raft_consensus.cc:482] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:31:09.635125 16964 raft_consensus.cc:504] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.635533 16964 leader_election.cc:288] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 760f2c4df2154a958eb10f08900cbfa2 (127.31.57.195:34167), 7b31d7a118124fe5babf21c3a30da414 (127.31.57.194:43335)
I20260809 19:31:09.638299 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.635750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:36971 (local address 127.31.57.195:34167)
0809 19:31:09.635808 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.635810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.635818 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:31:09.635847 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.635848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.635857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.635919 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:09.636082 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.636574 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.637556 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.637931 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.638017 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.638027 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.638184 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.638220 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:31:09.638303 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.635670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34167 (local address 127.31.57.193:36971)
0809 19:31:09.635714 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.635726 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.635773 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.635935 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.635936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.635937 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:09.636010 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.636012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.636660 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.636661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.637452 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.637454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.638015 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.638016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.638099 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.638103 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.638165 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.638208 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 19:31:09.638329 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.635843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59569 (local address 127.31.57.194:43335)
0809 19:31:09.635884 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.635886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.635920 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:31:09.635971 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.635972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.635980 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.636029 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:09.636193 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.636654 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.637595 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.637971 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.638055 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.638063 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.638226 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.638260 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:31:09.638412 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.635701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43335 (local address 127.31.57.193:59569)
0809 19:31:09.635869 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.635878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:09.635919 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:09.636036 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:09.636036 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:09.636038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:09.636106 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.636108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.636760 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.636761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.637494 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:09.637496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.638066 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.638067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:09.638147 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.638151 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:31:09.638218 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:31:09.638256 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":126,"threads_started":1}
I20260809 19:31:09.638654 16849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c674145374270bfc246ca5a9c979a" candidate_uuid: "34b60020e3714a9c9818a77a12b1968a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760f2c4df2154a958eb10f08900cbfa2" is_pre_election: true
I20260809 19:31:09.638806 16849 raft_consensus.cc:2454] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b60020e3714a9c9818a77a12b1968a in term 0.
I20260809 19:31:09.638782 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c674145374270bfc246ca5a9c979a" candidate_uuid: "34b60020e3714a9c9818a77a12b1968a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b31d7a118124fe5babf21c3a30da414" is_pre_election: true
I20260809 19:31:09.638945 16616 raft_consensus.cc:2454] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b60020e3714a9c9818a77a12b1968a in term 0.
I20260809 19:31:09.639067 16282 leader_election.cc:302] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b60020e3714a9c9818a77a12b1968a, 760f2c4df2154a958eb10f08900cbfa2; no voters:
I20260809 19:31:09.639206 16964 raft_consensus.cc:2781] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:31:09.639262 16964 raft_consensus.cc:482] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:31:09.639299 16964 raft_consensus.cc:3037] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:09.639999 16964 raft_consensus.cc:504] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.640156 16964 leader_election.cc:288] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [CANDIDATE]: Term 1 election: Requested vote from peers 760f2c4df2154a958eb10f08900cbfa2 (127.31.57.195:34167), 7b31d7a118124fe5babf21c3a30da414 (127.31.57.194:43335)
I20260809 19:31:09.640302 16616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c674145374270bfc246ca5a9c979a" candidate_uuid: "34b60020e3714a9c9818a77a12b1968a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b31d7a118124fe5babf21c3a30da414"
I20260809 19:31:09.640331 16849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7c674145374270bfc246ca5a9c979a" candidate_uuid: "34b60020e3714a9c9818a77a12b1968a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760f2c4df2154a958eb10f08900cbfa2"
I20260809 19:31:09.640383 16616 raft_consensus.cc:3037] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:09.640431 16849 raft_consensus.cc:3037] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:09.641110 16849 raft_consensus.cc:2454] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b60020e3714a9c9818a77a12b1968a in term 1.
I20260809 19:31:09.641142 16616 raft_consensus.cc:2454] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b60020e3714a9c9818a77a12b1968a in term 1.
I20260809 19:31:09.641326 16285 leader_election.cc:302] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b60020e3714a9c9818a77a12b1968a, 7b31d7a118124fe5babf21c3a30da414; no voters:
I20260809 19:31:09.641458 16964 raft_consensus.cc:2781] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:09.641621 16964 raft_consensus.cc:686] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [term 1 LEADER]: Becoming Leader. State: Replica: 34b60020e3714a9c9818a77a12b1968a, State: Running, Role: LEADER
I20260809 19:31:09.641721 16964 consensus_queue.cc:237] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } }
I20260809 19:31:09.642482 16156 catalog_manager.cc:5495] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b60020e3714a9c9818a77a12b1968a (127.31.57.193). New cstate: current_term: 1 leader_uuid: "34b60020e3714a9c9818a77a12b1968a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b60020e3714a9c9818a77a12b1968a" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 } health_report { overall_health: UNKNOWN } } }
I20260809 19:31:09.661430 16948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.659553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.31.57.193:39965)
0809 19:31:09.659591 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:09.659593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:09.659620 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:31:09.659680 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:09.659681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:09.659693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:09.659745 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:31:09.659865 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.660245 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.660864 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:09.661019 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:09.661063 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:09.661190 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:31:09.661271 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:31:09.661328 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:09.661361 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 19:31:09.661374 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:31:09.663100 16849 raft_consensus.cc:1261] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 1 FOLLOWER]: Refusing update from remote peer 34b60020e3714a9c9818a77a12b1968a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:31:09.663107 16616 raft_consensus.cc:1261] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 1 FOLLOWER]: Refusing update from remote peer 34b60020e3714a9c9818a77a12b1968a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:31:09.663473 16966 consensus_queue.cc:1035] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [LEADER]: Connected to new peer: Peer: permanent_uuid: "760f2c4df2154a958eb10f08900cbfa2" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:31:09.663614 16964 consensus_queue.cc:1035] T 9a7c674145374270bfc246ca5a9c979a P 34b60020e3714a9c9818a77a12b1968a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b31d7a118124fe5babf21c3a30da414" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:31:09.665741 16970 mvcc.cc:205] Tried to move back new op lower bound from 7316700650137288704 to 7316700650052988928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700650137288704}]
I20260809 19:31:09.665892 16971 mvcc.cc:205] Tried to move back new op lower bound from 7316700650137288704 to 7316700650052988928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700650137288704}]
I20260809 19:31:09.666663 16973 mvcc.cc:205] Tried to move back new op lower bound from 7316700650137288704 to 7316700650052988928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700650137288704}]
I20260809 19:31:09.668237 16157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:31:09.668337 16157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:31:09.669080 16157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37c3ed710aca4b5db92bba03ff6a3d86: Sending DeleteTablet for 3 replicas of tablet 9a7c674145374270bfc246ca5a9c979a
I20260809 19:31:09.669476 16588 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c674145374270bfc246ca5a9c979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:09 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:49072
I20260809 19:31:09.669714 16984 tablet_replica.cc:330] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: stopping tablet replica
I20260809 19:31:09.669727 16813 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c674145374270bfc246ca5a9c979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:09 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:59708
I20260809 19:31:09.669912 16984 raft_consensus.cc:2229] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:31:09.669976 16985 tablet_replica.cc:330] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: stopping tablet replica
I20260809 19:31:09.669998 16365 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a7c674145374270bfc246ca5a9c979a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:09 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:54694
I20260809 19:31:09.670079 16985 raft_consensus.cc:2229] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:31:09.670181 16985 raft_consensus.cc:2258] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:31:09.670313 16984 raft_consensus.cc:2258] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:31:09.670372 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 16254
I20260809 19:31:09.670610 16985 ts_tablet_manager.cc:1905] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:31:09.671427 16984 ts_tablet_manager.cc:1905] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:31:09.672556 16985 ts_tablet_manager.cc:1918] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:31:09.672622 16985 log.cc:1196] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/9a7c674145374270bfc246ca5a9c979a
I20260809 19:31:09.672847 16985 ts_tablet_manager.cc:1939] T 9a7c674145374270bfc246ca5a9c979a P 760f2c4df2154a958eb10f08900cbfa2: Deleting consensus metadata
I20260809 19:31:09.673583 16984 ts_tablet_manager.cc:1918] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:31:09.673689 16984 log.cc:1196] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/9a7c674145374270bfc246ca5a9c979a
I20260809 19:31:09.674108 16984 ts_tablet_manager.cc:1939] T 9a7c674145374270bfc246ca5a9c979a P 7b31d7a118124fe5babf21c3a30da414: Deleting consensus metadata
I20260809 19:31:09.674604 16130 catalog_manager.cc:4841] TS 7b31d7a118124fe5babf21c3a30da414 (127.31.57.194:43335): tablet 9a7c674145374270bfc246ca5a9c979a (table test-table [id=50c96df1620c4db99d6b0c3924e3916a]) successfully deleted
I20260809 19:31:09.675583 16120 catalog_manager.cc:4841] TS 760f2c4df2154a958eb10f08900cbfa2 (127.31.57.195:34167): tablet 9a7c674145374270bfc246ca5a9c979a (table test-table [id=50c96df1620c4db99d6b0c3924e3916a]) successfully deleted
W20260809 19:31:09.687122 16120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:39965 (error 108)
I20260809 19:31:09.687409 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 16486
I20260809 19:31:09.689687 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:09.687562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:39965 (local address 127.0.0.1:54706)
0809 19:31:09.689596 (+ 2034us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:09.689647 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:39965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2011}
W20260809 19:31:09.689854 16120 catalog_manager.cc:4584] TS 34b60020e3714a9c9818a77a12b1968a (127.31.57.193:39965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a7c674145374270bfc246ca5a9c979a: Network error: Client connection negotiation failed: client connection to 127.31.57.193:39965: connect: Connection refused (error 111)
I20260809 19:31:09.697530 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 16703
I20260809 19:31:09.709996 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 16094
2026-08-09T19:31:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2144 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16995](info): set up 2 sockets
Aug 09 19:31:09 dist-test-slave-x4qh krb5kdc[16995](info): commencing operation
krb5kdc: starting...
W20260809 19:31:10.894253 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:31:10 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:31:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangeradmin_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangeradmin_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangerlookup_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangerlookup_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
I20260809 19:31:11.043749 31975 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrXpU3x/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/postgres -l logfile start
2026-08-09 19:31:13.276 UTC [17045] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:31:13.276 UTC [17045] LOG: listening on IPv4 address "127.31.57.212", port 53893
2026-08-09 19:31:13.279 UTC [17045] LOG: listening on Unix socket "/tmp/.s.PGSQL.53893"
2026-08-09 19:31:13.298 UTC [17052] LOG: database system was shut down at 2026-08-09 19:31:12 UTC
2026-08-09 19:31:13.302 UTC [17045] LOG: database system is ready to accept connections
I20260809 19:31:14.404863 31975 mini_postgres.cc:96] Postgres bound to 53893
2026-08-09 19:31:14.413 UTC [17064] FATAL: database "slave" does not exist
127.31.57.212:53893 - accepting connections
I20260809 19:31:14.414438 31975 mini_ranger.cc:161] Starting Ranger...
I20260809 19:31:14.429549 31975 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:31:14.826531 31975 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:31:14.826599 31975 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin
2026-08-09 19:31:15,142 [I] DB FLAVOR :POSTGRES
2026-08-09 19:31:15,143 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:31:15,143 [I] Checking connection..
2026-08-09 19:31:15,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:31:15,400 [I] Checking connection passed.
2026-08-09 19:31:15,400 [I] --------- Verifying version history table ---------
2026-08-09 19:31:15,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:31:15,634 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:31:15,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:31:15,879 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:31:15,879 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:31:15,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:31:16.113 UTC [17161] WARNING: there is no transaction in progress
2026-08-09 19:31:16,119 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:31:16,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:31:16,350 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:31:16,350 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:31:16,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:31:16,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:31:16,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-09 19:31:17,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:31:17,292 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:31:17,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:31:17,548 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:31:17,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:31:17,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:31:18,022 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:31:18,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:31:18.900 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:18.916 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:18.930 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:18.943 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:18.960 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:18.979 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.033 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.039 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.052 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.061 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.069 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.082 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.090 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.096 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.101 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19.107 UTC [17350] WARNING: there is no transaction in progress
2026-08-09 19:31:19,681 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:31:19,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:31:19,926 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:31:19,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:31:20,179 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:31:20,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:31:20,424 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:31:20,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:31:20,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:31:20,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-09 19:31:21,149 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:31:21,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:31:21,378 [I] DB_PATCHES have already been applied
I20260809 19:31:21.384195 31975 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrXpU3x/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:31:21.384250 31975 mini_ranger.cc:193] Using host: 127.31.57.212
I20260809 19:31:21.388214 31975 mini_ranger.cc:227] Ranger admin URL: http://127.31.57.212:36379
Aug 09, 2026 7:31:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:31:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:31:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:31:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:31:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:31:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:31:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:31:22.680702 31975 mini_ranger.cc:160] Time spent starting Ranger: real 8.266s user 0.000s sys 0.010s
Aug 09, 2026 7:31:42 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:31:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 7:31:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:31:42 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:31:43 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:31:43.602123 31975 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangerkms_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/rangerkms_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.57.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.57.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:31:43 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303903, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:31:43.680428 31975 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:31:43.692920 31975 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:31:44.447170 31975 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:31:44.447237 31975 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms
I20260809 19:31:44.447248 31975 mini_ranger_kms.cc:227] Using postgres at 127.31.57.212:53893
2026-08-09 19:31:45,244 [I] DB FLAVOR :POSTGRES
2026-08-09 19:31:45,244 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:31:45,244 [I] Checking connection
2026-08-09 19:31:45,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:31:45,550 [I] connection success
2026-08-09 19:31:45,550 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:31:45,550 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:31:45,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:31:45,788 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:31:45,789 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:31:45,789 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:31:45,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:53893/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:31:46,042 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:31:46.454288 31975 mini_ranger_kms.cc:310] Created kms service
I20260809 19:31:46.628160 31975 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:31:46.676002 31975 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:31:46 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303903, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:47.076988 31975 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:31:47.077103 31975 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:31:47.077128 31975 mini_ranger_kms.cc:240] Using host: 127.31.57.212
I20260809 19:31:47.081342 31975 mini_ranger_kms.cc:276] Ranger KMS PID: 17659
I20260809 19:31:47.081459 31975 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.57.212:42667
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:31:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 19:31:49 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303909, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:49.852945 31975 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.173s user 0.000s sys 0.680s
I20260809 19:31:49.853057 31975 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:31:49.853132 31975 mini_ranger_kms.cc:401] 127.31.57.212:42667/kms/v1/keys
Aug 09 19:31:50 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303909, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
Aug 09 19:31:50 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303909, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
Aug 09 19:31:52 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:31:52.670079 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37755
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:37755
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:42667/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:52.769655 17754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:52.769845 17754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:52.769869 17754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:52.770989 17754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:31:52.771030 17754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:52.771047 17754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:52.771061 17754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:31:52.771075 17754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:31:52.772825 17754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.57.212:42667/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:37755
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:52.773011 17754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:52.773178 17754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:52.778600 17771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:52.778594 17767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:52.778575 17754 server_base.cc:1034] running on GCE node
W20260809 19:31:52.778563 17764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:52.779075 17754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:52.779417 17754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:52.780575 17754 hybrid_clock.cc:648] HybridClock initialized: now 1786303912780564 us; error 40 us; skew 500 ppm
Aug 09 19:31:52 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303912, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:52.783551 17754 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:31:52.784054 17754 webserver.cc:460] Webserver started at http://127.31.57.254:37331/ using document root <none> and password file <none>
I20260809 19:31:52.784329 17754 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:52.784404 17754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:52.784548 17754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:31:52 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303912, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:52.830438 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "88794074208e4539aadf3bd43c78c5d1"
format_stamp: "Formatted at 2026-08-09 19:31:52 on dist-test-slave-x4qh"
server_key: "26f4df01386eb6af29e12872c601dc09"
server_key_iv: "db09243ea8613532e135ce7ef69fce06"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:52.831007 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "88794074208e4539aadf3bd43c78c5d1"
format_stamp: "Formatted at 2026-08-09 19:31:52 on dist-test-slave-x4qh"
server_key: "26f4df01386eb6af29e12872c601dc09"
server_key_iv: "db09243ea8613532e135ce7ef69fce06"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:52.834254 17754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 19:31:52.849679 17788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:52.849929 17754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:31:52.850066 17754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "88794074208e4539aadf3bd43c78c5d1"
format_stamp: "Formatted at 2026-08-09 19:31:52 on dist-test-slave-x4qh"
server_key: "26f4df01386eb6af29e12872c601dc09"
server_key_iv: "db09243ea8613532e135ce7ef69fce06"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:52.850188 17754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:52.872491 17754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:52.874650 17754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:52.874850 17754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:52.882869 17754 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:37755
I20260809 19:31:52.882900 17870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:37755 every 8 connection(s)
I20260809 19:31:52.883288 17754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:31:52.884148 17872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:52.885699 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 17754
I20260809 19:31:52.885838 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:31:52.886024 31975 external_mini_cluster.cc:1428] Setting key 0cdef52b12449c8503cb0258ec2bf623
I20260809 19:31:52.887773 17872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: Bootstrap starting.
I20260809 19:31:52.888684 17872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:52.889289 17872 log.cc:824] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:52.890168 17872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: No bootstrap required, opened a new log
I20260809 19:31:52.892860 17872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } }
I20260809 19:31:52.893000 17872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:52.893024 17872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88794074208e4539aadf3bd43c78c5d1, State: Initialized, Role: FOLLOWER
I20260809 19:31:52.893175 17872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } }
I20260809 19:31:52.893253 17872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:31:52.893275 17872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:31:52.893335 17872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:31:52 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:52.894024 17872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } }
I20260809 19:31:52.894137 17872 leader_election.cc:302] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88794074208e4539aadf3bd43c78c5d1; no voters:
I20260809 19:31:52.894341 17872 leader_election.cc:288] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:31:52.894381 17880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:52.894795 17872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:31:52.895201 17880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [term 1 LEADER]: Becoming Leader. State: Replica: 88794074208e4539aadf3bd43c78c5d1, State: Running, Role: LEADER
I20260809 19:31:52.895319 17880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } }
I20260809 19:31:52.896353 17880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88794074208e4539aadf3bd43c78c5d1. Latest consensus state: current_term: 1 leader_uuid: "88794074208e4539aadf3bd43c78c5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } } }
I20260809 19:31:52.896453 17880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:52.896392 17885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88794074208e4539aadf3bd43c78c5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88794074208e4539aadf3bd43c78c5d1" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37755 } } }
I20260809 19:31:52.896519 17885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:31:52.897921 17872 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:31:52.898454 17895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:31:52.902451 17895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:31:52.904475 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:52.886557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.31.57.254:37755)
0809 19:31:52.886844 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:52.886853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:52.886903 (+ 50us) server_negotiation.cc:407] Connection header received
0809 19:31:52.887622 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:52.887631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:52.887689 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:52.887832 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:52.888534 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:52.890063 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:52.890893 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:52.891664 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:52.891779 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:52.894577 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:52.894580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:52.894581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:52.894591 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:52.897822 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:52.898238 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:52.898241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:52.898242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:52.898368 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:52.898722 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:52.898723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:52.898724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:52.901447 (+ 2723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:52.902490 (+ 1043us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:52.902676 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 19:31:52.903742 (+ 1066us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":117,"threads_started":1}
I20260809 19:31:52.906548 17895 catalog_manager.cc:1347] Generated new cluster ID: 5514b50894684e0589b08a759efc9a24
I20260809 19:31:52.906618 17895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:31:52.912635 17895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:31:52.913446 17895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:31:52.920696 17895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: Generated new TSK 0
I20260809 19:31:52.920929 17895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:31:53.708578 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:42667/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:53.828294 17947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:53.828506 17947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:53.828544 17947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:53.829749 17947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:53.829815 17947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:53.829854 17947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:31:53.831425 17947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.57.212:42667/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:53.831674 17947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:53.831912 17947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:53.837206 17956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:53.837479 17962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:53.837143 17955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:53.839040 17947 server_base.cc:1034] running on GCE node
I20260809 19:31:53.839327 17947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:53.839773 17947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:53.840973 17947 hybrid_clock.cc:648] HybridClock initialized: now 1786303913840948 us; error 38 us; skew 500 ppm
Aug 09 19:31:53 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303913, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:53.848831 17947 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:31:53.849296 17947 webserver.cc:460] Webserver started at http://127.31.57.193:45543/ using document root <none> and password file <none>
I20260809 19:31:53.849534 17947 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:53.849599 17947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:53.849727 17947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:31:53 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303913, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:53.867733 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "beb48b4d1b784d65aa69a4b57a232821"
format_stamp: "Formatted at 2026-08-09 19:31:53 on dist-test-slave-x4qh"
server_key: "5ee5615596a3ed0133a0933a85fe2263"
server_key_iv: "b5b38dd99423ecf035847ae34984f6fa"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:53.868256 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "beb48b4d1b784d65aa69a4b57a232821"
format_stamp: "Formatted at 2026-08-09 19:31:53 on dist-test-slave-x4qh"
server_key: "5ee5615596a3ed0133a0933a85fe2263"
server_key_iv: "b5b38dd99423ecf035847ae34984f6fa"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:53.870580 17947 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 19:31:53.890945 17975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:53.891234 17947 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260809 19:31:53.891361 17947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "beb48b4d1b784d65aa69a4b57a232821"
format_stamp: "Formatted at 2026-08-09 19:31:53 on dist-test-slave-x4qh"
server_key: "5ee5615596a3ed0133a0933a85fe2263"
server_key_iv: "b5b38dd99423ecf035847ae34984f6fa"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:53.891508 17947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:53.910435 17947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:53.912581 17947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:53.912771 17947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:53.913046 17947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:53.913376 17947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:53.913431 17947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:53.913470 17947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:53.913513 17947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:53.928470 17947 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:43999
I20260809 19:31:53.928671 18179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:43999 every 8 connection(s)
I20260809 19:31:53.928930 17947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:31:53.933122 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 17947
I20260809 19:31:53.933221 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:31:53.933419 31975 external_mini_cluster.cc:1428] Setting key 74cf4b7fbc89c72b198ab910afd40849
Aug 09 19:31:53 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303913, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:53.940999 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:53.929796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37755 (local address 127.31.57.193:34263)
0809 19:31:53.930141 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:53.930185 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:53.930994 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:53.931493 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:53.931497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:53.931771 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:53.931970 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:53.931975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:53.932963 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:53.932964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:53.934165 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:53.934169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:53.934581 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:53.934582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:53.934718 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:53.935261 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:53.935269 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:53.936977 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:53.939139 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:53.939146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:53.939148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:53.939501 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:53.939757 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:53.939758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:53.939758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:53.939901 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:53.940276 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:53.940283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:53.940443 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 19:31:53.940550 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 19:31:53.940840 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":193,"threads_started":1}
I20260809 19:31:53.941412 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:53.929900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:34263 (local address 127.31.57.254:37755)
0809 19:31:53.931231 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:53.931234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:53.931242 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:31:53.931270 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:53.931270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:53.931283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:53.931386 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:53.932078 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:53.932824 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:53.934270 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:53.934462 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:53.934542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:53.937101 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:53.937102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:53.937103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:53.937112 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:53.939004 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:53.939606 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:53.939607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:53.939608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:53.939681 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:53.939984 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:53.939985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:53.939986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:53.940169 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:53.940242 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:53.941227 (+ 985us) server_negotiation.cc:299] Negotiation successful
0809 19:31:53.941321 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":99,"threads_started":1}
I20260809 19:31:53.941740 18180 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37755
I20260809 19:31:53.941825 18180 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:53.942011 18180 heartbeater.cc:511] Master 127.31.57.254:37755 requested a full tablet report, sending...
I20260809 19:31:53.942526 17813 ts_manager.cc:194] Registered new tserver with Master: beb48b4d1b784d65aa69a4b57a232821 (127.31.57.193:43999)
I20260809 19:31:53.943428 17813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:34263
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:31:53.967535 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:42667/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:54.119530 18190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:54.119836 18190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:54.119877 18190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:54.121709 18190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:54.121832 18190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:54.121876 18190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:31:54.124208 18190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.57.212:42667/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:54.124495 18190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:54.124797 18190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:54.132051 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:54.132128 18200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:54.132503 18205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:54.132952 18190 server_base.cc:1034] running on GCE node
I20260809 19:31:54.133162 18190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:54.133430 18190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:54.134613 18190 hybrid_clock.cc:648] HybridClock initialized: now 1786303914134582 us; error 44 us; skew 500 ppm
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:54.138486 18190 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:31:54.138989 18190 webserver.cc:460] Webserver started at http://127.31.57.194:36865/ using document root <none> and password file <none>
I20260809 19:31:54.139261 18190 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:54.139331 18190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:54.139511 18190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:54.154235 18190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "b4ed6eecd79e472bbc44ddd120425eb4"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "e376d28050a3d0daf0e9bd83efb72321"
server_key_iv: "364a771b539486e60d2ef063489e9e9c"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.154691 18190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "b4ed6eecd79e472bbc44ddd120425eb4"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "e376d28050a3d0daf0e9bd83efb72321"
server_key_iv: "364a771b539486e60d2ef063489e9e9c"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.159864 18190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 19:31:54.176535 18214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:54.185918 18190 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260809 19:31:54.186035 18190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "b4ed6eecd79e472bbc44ddd120425eb4"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "e376d28050a3d0daf0e9bd83efb72321"
server_key_iv: "364a771b539486e60d2ef063489e9e9c"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.186131 18190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:54.201565 18190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:54.203831 18190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:54.204043 18190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:54.204353 18190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:54.204926 18190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:54.205032 18190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:54.205106 18190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:54.205161 18190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:54.218042 18190 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:35173
I20260809 19:31:54.218509 18190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:31:54.219720 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 18190
I20260809 19:31:54.219815 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:31:54.220232 31975 external_mini_cluster.cc:1428] Setting key c95cf8aa7a89faf0dac397a9c59d090b
I20260809 19:31:54.226667 18407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:35173 every 8 connection(s)
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
I20260809 19:31:54.240298 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.228009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:36563 (local address 127.31.57.254:37755)
0809 19:31:54.228068 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.228070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.229529 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 19:31:54.230169 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.230170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.230215 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.230281 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:54.231408 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.232144 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.233676 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.233969 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.234014 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.236765 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:54.236766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:54.236767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:54.236773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:54.238567 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.239096 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.239098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.239098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.239157 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.239496 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.239497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.239498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.239636 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:54.239697 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.240071 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.240172 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:31:54.241040 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.227704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37755 (local address 127.31.57.194:36563)
0809 19:31:54.229399 (+ 1695us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.229446 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.230045 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.230433 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.230438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.230848 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:54.231246 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.231255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.232262 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.232264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.233566 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.233569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.234106 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.234107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.234270 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.234921 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:54.234930 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:54.236635 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:54.238683 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:54.238691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:54.238693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:54.238982 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:54.239245 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:54.239246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:54.239246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:54.239367 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:54.239752 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:54.239759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:54.239913 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.240631 (+ 718us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.240821 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1649,"thread_start_us":99,"threads_started":1}
I20260809 19:31:54.241678 18416 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37755
I20260809 19:31:54.241767 18416 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:54.241940 18416 heartbeater.cc:511] Master 127.31.57.254:37755 requested a full tablet report, sending...
I20260809 19:31:54.242343 17813 ts_manager.cc:194] Registered new tserver with Master: b4ed6eecd79e472bbc44ddd120425eb4 (127.31.57.194:35173)
I20260809 19:31:54.242796 17813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:36563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:31:54.264142 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:42667/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:31:54.388686 18425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:31:54.388989 18425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:31:54.389068 18425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:31:54.390712 18425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:31:54.390827 18425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:31:54.390901 18425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:31:54.392637 18425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.57.212:42667/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:31:54.392874 18425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:31:54.393136 18425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:31:54.411720 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:54.412658 18438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:31:54.419251 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:31:54.420302 18425 server_base.cc:1034] running on GCE node
I20260809 19:31:54.420601 18425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:31:54.420910 18425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:31:54.422097 18425 hybrid_clock.cc:648] HybridClock initialized: now 1786303914422069 us; error 38 us; skew 500 ppm
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:31:54.426388 18425 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:31:54.427007 18425 webserver.cc:460] Webserver started at http://127.31.57.195:40497/ using document root <none> and password file <none>
I20260809 19:31:54.427291 18425 fs_manager.cc:359] Metadata directory not provided
I20260809 19:31:54.427372 18425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:31:54.427542 18425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:54.452479 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "0ec2670223f9358a6239723f1f460e32"
server_key_iv: "cbecc72c24d6ed6bea79f5862b00bec0"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.452929 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "0ec2670223f9358a6239723f1f460e32"
server_key_iv: "cbecc72c24d6ed6bea79f5862b00bec0"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.456439 18425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 19:31:54.472837 18450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:54.473073 18425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:31:54.473176 18425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507"
format_stamp: "Formatted at 2026-08-09 19:31:54 on dist-test-slave-x4qh"
server_key: "0ec2670223f9358a6239723f1f460e32"
server_key_iv: "cbecc72c24d6ed6bea79f5862b00bec0"
server_key_version: "kuduclusterkey@0"
I20260809 19:31:54.473277 18425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:31:54.481003 18425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:31:54.483230 18425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:31:54.483429 18425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:31:54.483661 18425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:31:54.484071 18425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:31:54.484141 18425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:54.484180 18425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:31:54.484210 18425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:31:54.498081 18425 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:45405
I20260809 19:31:54.498570 18425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:31:54.500219 18637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:45405 every 8 connection(s)
I20260809 19:31:54.501982 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 18425
I20260809 19:31:54.502094 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:31:54.502293 31975 external_mini_cluster.cc:1428] Setting key 24e84d2809d31fa048135815356c2418
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:31:54.514731 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.502051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:56329 (local address 127.31.57.254:37755)
0809 19:31:54.502098 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.502100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.502555 (+ 455us) server_negotiation.cc:407] Connection header received
0809 19:31:54.503611 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.503612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.503625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.503657 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:54.504895 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.505906 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.507680 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.508027 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.508086 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.511290 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:54.511292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:54.511293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:54.511300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:54.513058 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.513580 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.513581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.513581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.513649 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.513954 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.513956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.513956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.514107 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:54.514158 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.514536 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.514623 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:31:54.515679 18643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.501585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37755 (local address 127.31.57.195:56329)
0809 19:31:54.502250 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.502294 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.503464 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.503909 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.503914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.504409 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:31:54.504786 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.504793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.506060 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.506061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.507553 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.507557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.508210 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.508211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.508363 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.509020 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:31:54.509048 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:31:54.511168 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:31:54.513158 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:54.513166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:54.513168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:54.513481 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:54.513734 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:31:54.513735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:31:54.513735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:31:54.513873 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:31:54.514215 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:31:54.514222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:31:54.514370 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.515178 (+ 808us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.515354 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":233,"threads_started":1}
I20260809 19:31:54.516456 18639 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37755
I20260809 19:31:54.516558 18639 heartbeater.cc:464] Registering TS with master...
I20260809 19:31:54.516742 18639 heartbeater.cc:511] Master 127.31.57.254:37755 requested a full tablet report, sending...
I20260809 19:31:54.517166 17813 ts_manager.cc:194] Registered new tserver with Master: 7b0ecbdaf9f9441f8b2c21cf4106a507 (127.31.57.195:45405)
I20260809 19:31:54.517663 17813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:56329
I20260809 19:31:54.524653 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:31:54.540447 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.526272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.31.57.254:37755)
0809 19:31:54.526305 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.526307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.531618 (+ 5311us) server_negotiation.cc:407] Connection header received
0809 19:31:54.531756 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.531757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.531767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.531793 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:31:54.532531 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.533213 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.534426 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.534694 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.534734 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.537744 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:31:54.537746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:31:54.537746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:31:54.537752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:31:54.539272 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.539674 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.539676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.539676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.539728 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:31:54.539901 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:31:54.539902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:31:54.539902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:31:54.540005 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:31:54.540076 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.540291 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.540363 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:31:54.542729 17813 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:31:54.545023 17813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:31:54.555789 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.548652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43999 (local address 127.0.0.1:35534)
0809 19:31:54.550356 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.550394 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.550463 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.551136 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.551139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.551141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:54.551198 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.551199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.553018 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.553019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.554279 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.554281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.555367 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.555397 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.555525 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.555537 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.555659 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.555717 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1644,"mutex_wait_us":3,"thread_start_us":1403,"threads_started":1}
I20260809 19:31:54.555796 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.549161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.31.57.193:43999)
0809 19:31:54.550170 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.550175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.550825 (+ 650us) server_negotiation.cc:407] Connection header received
0809 19:31:54.550864 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.550866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.550884 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.551052 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:54.551742 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.552905 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.554390 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.555180 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.555258 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.555314 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.555679 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.555720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":59,"threads_started":1}
I20260809 19:31:54.556887 18057 tablet_service.cc:1467] Processing CreateTablet for tablet 45037e60e1bc4e7d8341a989a02db499 (DEFAULT_TABLE table=test-table [id=d9ebc68a534f4f6e9c4de08295455b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:54.557320 18057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45037e60e1bc4e7d8341a989a02db499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:54.559907 18657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.548803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:35173 (local address 127.0.0.1:54676)
0809 19:31:54.551599 (+ 2796us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.551608 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.551719 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.551988 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.551990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.551993 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:54.552036 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.552038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.553291 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.553292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.554277 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.554280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.559700 (+ 5420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.559700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.559749 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.559752 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.559799 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.559846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2774,"thread_start_us":72,"threads_started":1}
I20260809 19:31:54.560366 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.549310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.31.57.194:35173)
0809 19:31:54.550262 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.550265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.551777 (+ 1512us) server_negotiation.cc:407] Connection header received
0809 19:31:54.551853 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.551856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.551871 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.551964 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:54.552167 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.553196 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.554446 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.555186 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.555258 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.555429 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.560197 (+ 4768us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.560260 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":89,"threads_started":1}
I20260809 19:31:54.561196 18297 tablet_service.cc:1467] Processing CreateTablet for tablet 45037e60e1bc4e7d8341a989a02db499 (DEFAULT_TABLE table=test-table [id=d9ebc68a534f4f6e9c4de08295455b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:54.561681 18297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45037e60e1bc4e7d8341a989a02db499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:54.562052 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.548652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:45405 (local address 127.0.0.1:35980)
0809 19:31:54.550607 (+ 1955us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.550615 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.550768 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.552580 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.552581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.552583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:54.552693 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.552695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.554735 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.554736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.559629 (+ 4893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.559633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.561817 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.561818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.561904 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.561909 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.561938 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.561973 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1942,"mutex_wait_us":2,"thread_start_us":1085,"threads_started":1}
I20260809 19:31:54.562623 18665 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Bootstrap starting.
I20260809 19:31:54.562827 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.548818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.31.57.195:45405)
0809 19:31:54.552198 (+ 3380us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.552203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.552212 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:31:54.552251 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.552252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.552268 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.552771 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:54.552859 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.554603 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.560682 (+ 6079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.561666 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.562396 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.562570 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.562600 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.562642 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3343,"thread_start_us":273,"threads_started":1}
I20260809 19:31:54.563640 18501 tablet_service.cc:1467] Processing CreateTablet for tablet 45037e60e1bc4e7d8341a989a02db499 (DEFAULT_TABLE table=test-table [id=d9ebc68a534f4f6e9c4de08295455b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:31:54.563967 18501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45037e60e1bc4e7d8341a989a02db499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:31:54.564538 18665 tablet_bootstrap.cc:654] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:54.565299 18665 log.cc:824] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:54.566421 18665 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: No bootstrap required, opened a new log
I20260809 19:31:54.566495 18665 ts_tablet_manager.cc:1397] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 19:31:54.567413 18672 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Bootstrap starting.
I20260809 19:31:54.569029 18672 tablet_bootstrap.cc:654] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:54.569886 18672 log.cc:824] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:54.570485 18673 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Bootstrap starting.
I20260809 19:31:54.571326 18665 raft_consensus.cc:348] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.571972 18665 raft_consensus.cc:374] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:54.572023 18665 raft_consensus.cc:729] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb48b4d1b784d65aa69a4b57a232821, State: Initialized, Role: FOLLOWER
I20260809 19:31:54.572188 18665 consensus_queue.cc:260] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.572307 18673 tablet_bootstrap.cc:654] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Neither blocks nor log segments found. Creating new log.
I20260809 19:31:54.572608 18665 ts_tablet_manager.cc:1428] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 19:31:54.572788 18180 heartbeater.cc:503] Master 127.31.57.254:37755 was elected leader, sending a full tablet report...
I20260809 19:31:54.572950 18673 log.cc:824] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Log is configured to *not* fsync() on all Append() calls
I20260809 19:31:54.573993 18672 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: No bootstrap required, opened a new log
I20260809 19:31:54.574069 18672 ts_tablet_manager.cc:1397] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 19:31:54.574393 18673 tablet_bootstrap.cc:492] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: No bootstrap required, opened a new log
I20260809 19:31:54.574481 18673 ts_tablet_manager.cc:1397] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 19:31:54.577517 18672 raft_consensus.cc:348] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.577728 18672 raft_consensus.cc:374] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:54.577796 18672 raft_consensus.cc:729] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ed6eecd79e472bbc44ddd120425eb4, State: Initialized, Role: FOLLOWER
I20260809 19:31:54.577970 18672 consensus_queue.cc:260] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.578300 18672 ts_tablet_manager.cc:1428] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:31:54.578248 18673 raft_consensus.cc:348] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.578378 18673 raft_consensus.cc:374] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:31:54.578411 18416 heartbeater.cc:503] Master 127.31.57.254:37755 was elected leader, sending a full tablet report...
I20260809 19:31:54.578424 18673 raft_consensus.cc:729] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0ecbdaf9f9441f8b2c21cf4106a507, State: Initialized, Role: FOLLOWER
I20260809 19:31:54.578845 18673 consensus_queue.cc:260] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.579214 18673 ts_tablet_manager.cc:1428] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 19:31:54.579330 18639 heartbeater.cc:503] Master 127.31.57.254:37755 was elected leader, sending a full tablet report...
W20260809 19:31:54.679998 18181 tablet.cc:2369] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:31:54.727533 18420 tablet.cc:2369] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:31:54.751521 18641 tablet.cc:2369] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:31:54.770741 18679 raft_consensus.cc:482] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:31:54.770879 18679 raft_consensus.cc:504] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.771395 18679 leader_election.cc:288] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers beb48b4d1b784d65aa69a4b57a232821 (127.31.57.193:43999), 7b0ecbdaf9f9441f8b2c21cf4106a507 (127.31.57.195:45405)
I20260809 19:31:54.788609 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.771739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:45585 (local address 127.31.57.195:45405)
0809 19:31:54.771785 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.771788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.776795 (+ 5007us) server_negotiation.cc:407] Connection header received
0809 19:31:54.777158 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.777160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.777174 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.777210 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:54.777933 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.778614 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.786618 (+ 8004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.787181 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.787241 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.787251 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.788470 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.788518 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:31:54.789153 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.772573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:44149 (local address 127.31.57.193:43999)
0809 19:31:54.772731 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.772733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.777085 (+ 4352us) server_negotiation.cc:407] Connection header received
0809 19:31:54.777521 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.777522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.777533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.777565 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:31:54.781094 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.781839 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.787551 (+ 5712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.788152 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.788210 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.788219 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.789021 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.789087 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:31:54.789609 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.771677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:45405 (local address 127.31.57.194:45585)
0809 19:31:54.776709 (+ 5032us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.776721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.776811 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.777351 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.777351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.777353 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:54.777426 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.777429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.778981 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.778982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.780150 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.780154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.787323 (+ 7169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.787324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.787435 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.787440 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.789450 (+ 2010us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.789502 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5013,"thread_start_us":227,"threads_started":1}
I20260809 19:31:54.789950 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.772162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43999 (local address 127.31.57.194:44149)
0809 19:31:54.776899 (+ 4737us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.776908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:31:54.777261 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:31:54.777717 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:31:54.777753 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:31:54.777756 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:31:54.777829 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.777831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.781957 (+ 4126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.781958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.783110 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:31:54.783113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.788287 (+ 5174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.788287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:31:54.788369 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.788372 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:31:54.789795 (+ 1423us) client_negotiation.cc:240] Negotiation successful
0809 19:31:54.789840 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4723,"thread_start_us":257,"threads_started":1}
I20260809 19:31:54.790422 18558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45037e60e1bc4e7d8341a989a02db499" candidate_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" is_pre_election: true
I20260809 19:31:54.790602 18558 raft_consensus.cc:2454] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ed6eecd79e472bbc44ddd120425eb4 in term 0.
I20260809 19:31:54.790798 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45037e60e1bc4e7d8341a989a02db499" candidate_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb48b4d1b784d65aa69a4b57a232821" is_pre_election: true
I20260809 19:31:54.790967 18097 raft_consensus.cc:2454] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ed6eecd79e472bbc44ddd120425eb4 in term 0.
I20260809 19:31:54.791250 18216 leader_election.cc:302] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b0ecbdaf9f9441f8b2c21cf4106a507, b4ed6eecd79e472bbc44ddd120425eb4; no voters:
I20260809 19:31:54.791435 18679 raft_consensus.cc:2781] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:31:54.791486 18679 raft_consensus.cc:482] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:31:54.791508 18679 raft_consensus.cc:3037] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:54.792658 18679 raft_consensus.cc:504] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.792783 18679 leader_election.cc:288] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [CANDIDATE]: Term 1 election: Requested vote from peers beb48b4d1b784d65aa69a4b57a232821 (127.31.57.193:43999), 7b0ecbdaf9f9441f8b2c21cf4106a507 (127.31.57.195:45405)
I20260809 19:31:54.793089 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45037e60e1bc4e7d8341a989a02db499" candidate_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb48b4d1b784d65aa69a4b57a232821"
I20260809 19:31:54.793160 18097 raft_consensus.cc:3037] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:31:54 dist-test-slave-x4qh krb5kdc[16995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303914, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:31:54.794489 18097 raft_consensus.cc:2454] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ed6eecd79e472bbc44ddd120425eb4 in term 1.
I20260809 19:31:54.794804 18221 leader_election.cc:302] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4ed6eecd79e472bbc44ddd120425eb4, beb48b4d1b784d65aa69a4b57a232821; no voters:
I20260809 19:31:54.795240 18558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45037e60e1bc4e7d8341a989a02db499" candidate_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507"
I20260809 19:31:54.795322 18558 raft_consensus.cc:3037] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:31:54.796478 18558 raft_consensus.cc:2454] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ed6eecd79e472bbc44ddd120425eb4 in term 1.
I20260809 19:31:54.796741 18679 raft_consensus.cc:2781] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:31:54.797056 18679 raft_consensus.cc:686] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 1 LEADER]: Becoming Leader. State: Replica: b4ed6eecd79e472bbc44ddd120425eb4, State: Running, Role: LEADER
I20260809 19:31:54.797147 18679 consensus_queue.cc:237] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } }
I20260809 19:31:54.797992 17812 catalog_manager.cc:5495] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4ed6eecd79e472bbc44ddd120425eb4 (127.31.57.194). New cstate: current_term: 1 leader_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4ed6eecd79e472bbc44ddd120425eb4" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 35173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 } health_report { overall_health: UNKNOWN } } }
I20260809 19:31:54.858520 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.855300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.31.57.194:35173)
0809 19:31:54.855339 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:31:54.855341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:31:54.855347 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:31:54.855410 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:31:54.855410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:31:54.855434 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:31:54.855494 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:31:54.855677 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.856346 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.857463 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:31:54.857768 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:31:54.857833 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:31:54.858047 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:31:54.858170 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:31:54.858250 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 19:31:54.858317 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:31:54.858338 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:31:54.860826 18558 raft_consensus.cc:1261] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 1 FOLLOWER]: Refusing update from remote peer b4ed6eecd79e472bbc44ddd120425eb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:31:54.861338 18686 consensus_queue.cc:1035] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b0ecbdaf9f9441f8b2c21cf4106a507" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:31:54.863402 18097 raft_consensus.cc:1261] T 45037e60e1bc4e7d8341a989a02db499 P beb48b4d1b784d65aa69a4b57a232821 [term 1 FOLLOWER]: Refusing update from remote peer b4ed6eecd79e472bbc44ddd120425eb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:31:54.863755 18686 consensus_queue.cc:1035] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb48b4d1b784d65aa69a4b57a232821" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:31:54.869139 18694 mvcc.cc:205] Tried to move back new op lower bound from 7316700835266891776 to 7316700835009691648. Current Snapshot: MvccSnapshot[applied={T|T < 7316700835266891776}]
I20260809 19:31:54.870359 18697 mvcc.cc:205] Tried to move back new op lower bound from 7316700835266891776 to 7316700835009691648. Current Snapshot: MvccSnapshot[applied={T|T < 7316700835266891776}]
I20260809 19:31:54.871930 17812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:31:54.872018 17812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:31:54.873081 17812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88794074208e4539aadf3bd43c78c5d1: Sending DeleteTablet for 3 replicas of tablet 45037e60e1bc4e7d8341a989a02db499
I20260809 19:31:54.873481 18057 tablet_service.cc:1514] Processing DeleteTablet for tablet 45037e60e1bc4e7d8341a989a02db499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:54 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:35534
I20260809 19:31:54.873904 18297 tablet_service.cc:1514] Processing DeleteTablet for tablet 45037e60e1bc4e7d8341a989a02db499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:54 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:54676
I20260809 19:31:54.874020 18501 tablet_service.cc:1514] Processing DeleteTablet for tablet 45037e60e1bc4e7d8341a989a02db499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:31:54 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:35980
I20260809 19:31:54.874213 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 17947
I20260809 19:31:54.883913 18706 tablet_replica.cc:330] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: stopping tablet replica
I20260809 19:31:54.884032 18706 raft_consensus.cc:2229] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:31:54.884122 18706 raft_consensus.cc:2258] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:31:54.884585 18706 ts_tablet_manager.cc:1905] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:31:54.887570 18705 tablet_replica.cc:330] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: stopping tablet replica
I20260809 19:31:54.887748 18705 raft_consensus.cc:2229] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:31:54.887908 18705 raft_consensus.cc:2258] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:31:54.888422 18705 ts_tablet_manager.cc:1905] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:31:54.890493 18705 ts_tablet_manager.cc:1918] T 45037e60e1bc4e7d8341a989a02db499 P b4ed6eecd79e472bbc44ddd120425eb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:31:54.891181 18706 ts_tablet_manager.cc:1918] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:31:54.891316 18706 log.cc:1196] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/45037e60e1bc4e7d8341a989a02db499
I20260809 19:31:54.892055 18706 ts_tablet_manager.cc:1939] T 45037e60e1bc4e7d8341a989a02db499 P 7b0ecbdaf9f9441f8b2c21cf4106a507: Deleting consensus metadata
I20260809 19:31:54.892606 17792 catalog_manager.cc:4841] TS 7b0ecbdaf9f9441f8b2c21cf4106a507 (127.31.57.195:45405): tablet 45037e60e1bc4e7d8341a989a02db499 (table test-table [id=d9ebc68a534f4f6e9c4de08295455b46]) successfully deleted
W20260809 19:31:54.892872 17795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:43999 (error 108)
I20260809 19:31:54.893577 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 18190
I20260809 19:31:54.904404 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 18425
I20260809 19:31:54.906118 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.904558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:35173 (local address 127.0.0.1:54702)
0809 19:31:54.906034 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.906079 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.194:35173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1454}
I20260809 19:31:54.906229 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:31:54.904582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43999 (local address 127.0.0.1:35542)
0809 19:31:54.906191 (+ 1609us) negotiation.cc:107] Waiting for socket to connect
0809 19:31:54.906210 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1597}
W20260809 19:31:54.912052 17793 catalog_manager.cc:4584] TS b4ed6eecd79e472bbc44ddd120425eb4 (127.31.57.194:35173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45037e60e1bc4e7d8341a989a02db499: Network error: Client connection negotiation failed: client connection to 127.31.57.194:35173: connect: Connection refused (error 111)
I20260809 19:31:54.917776 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 17754
I20260809 19:31:54.928694 31975 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:31:55.273475 31975 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:31:55.273602 31975 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:31:55.618556 31975 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:31:55.618 UTC [17045] LOG: received smart shutdown request
2026-08-09 19:31:55.622 UTC [17045] LOG: background worker "logical replication launcher" (PID 17061) exited with exit code 1
2026-08-09 19:31:55.622 UTC [17054] LOG: shutting down
2026-08-09 19:31:55.754 UTC [17045] LOG: database system is shut down
2026-08-09T19:31:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46149 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:31:55 dist-test-slave-x4qh krb5kdc[18720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:31:55 dist-test-slave-x4qh krb5kdc[18720](info): set up 2 sockets
Aug 09 19:31:55 dist-test-slave-x4qh krb5kdc[18720](info): commencing operation
krb5kdc: starting...
W20260809 19:31:57.070822 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:31:57 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:31:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:31:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangeradmin_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangeradmin_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangerlookup_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangerlookup_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
I20260809 19:31:57.218717 31975 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrXpU3x/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/postgres -l logfile start
2026-08-09 19:31:59.383 UTC [18775] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:31:59.383 UTC [18775] LOG: listening on IPv4 address "127.31.57.212", port 43435
2026-08-09 19:31:59.386 UTC [18775] LOG: listening on Unix socket "/tmp/.s.PGSQL.43435"
2026-08-09 19:31:59.405 UTC [18781] LOG: database system was shut down at 2026-08-09 19:31:58 UTC
2026-08-09 19:31:59.408 UTC [18775] LOG: database system is ready to accept connections
I20260809 19:32:00.497922 31975 mini_postgres.cc:96] Postgres bound to 43435
2026-08-09 19:32:00.505 UTC [18791] FATAL: database "slave" does not exist
127.31.57.212:43435 - accepting connections
I20260809 19:32:00.505513 31975 mini_ranger.cc:161] Starting Ranger...
I20260809 19:32:00.519250 31975 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:32:00.904459 31975 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:32:00.904536 31975 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin
2026-08-09 19:32:01,157 [I] DB FLAVOR :POSTGRES
2026-08-09 19:32:01,158 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:32:01,158 [I] Checking connection..
2026-08-09 19:32:01,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:32:01,402 [I] Checking connection passed.
2026-08-09 19:32:01,402 [I] --------- Verifying version history table ---------
2026-08-09 19:32:01,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:32:01,631 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:32:01,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:32:01,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:32:01,878 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:32:01,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:32:02.120 UTC [18903] WARNING: there is no transaction in progress
2026-08-09 19:32:02,128 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:32:02,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:32:02,375 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:32:02,375 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:32:02,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:32:02,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:32:02,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-09 19:32:03,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:32:03,290 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:32:03,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:32:03,534 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:32:03,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:32:03,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:32:04,004 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:32:04,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:32:04.821 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.835 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.849 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.862 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.876 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.892 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.948 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.953 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.966 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.976 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.985 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:04.997 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:05.005 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:05.011 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:05.017 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:05.023 UTC [19074] WARNING: there is no transaction in progress
2026-08-09 19:32:05,560 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:32:05,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:32:05,793 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:32:05,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:32:06,028 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:32:06,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:32:06,259 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:32:06,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:32:06,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:32:06,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-09 19:32:06,949 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:32:06,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:32:07,185 [I] DB_PATCHES have already been applied
I20260809 19:32:07.190706 31975 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrXpU3x/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:32:07.190757 31975 mini_ranger.cc:193] Using host: 127.31.57.212
I20260809 19:32:07.194363 31975 mini_ranger.cc:227] Ranger admin URL: http://127.31.57.212:39391
Aug 09, 2026 7:32:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:32:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:32:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:32:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:32:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:32:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:32:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:32:08.518385 31975 mini_ranger.cc:160] Time spent starting Ranger: real 8.013s user 0.000s sys 0.009s
Aug 09, 2026 7:32:26 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:32:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 7:32:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:32:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:32:27 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:32:28.263348 31975 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.57.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.57.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangerkms_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.57.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/rangerkms_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.57.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.57.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.57.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/HTTP_127.31.57.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:32:28 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303948, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:32:28.344043 31975 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:32:28.355337 31975 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:32:29.108649 31975 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:32:29.108718 31975 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms
I20260809 19:32:29.108731 31975 mini_ranger_kms.cc:227] Using postgres at 127.31.57.212:43435
2026-08-09 19:32:29,302 [I] DB FLAVOR :POSTGRES
2026-08-09 19:32:29,302 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:32:29,302 [I] Checking connection
2026-08-09 19:32:29,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:32:29,526 [I] connection success
2026-08-09 19:32:29,526 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:32:29,527 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:32:29,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:32:29,752 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:32:29,753 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:32:29,753 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:32:29,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.57.212:43435/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:32:30,007 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:32:30.444401 31975 mini_ranger_kms.cc:310] Created kms service
I20260809 19:32:30.631801 31975 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:32:30.672592 31975 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:32:30 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303948, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:31.035333 31975 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:32:31.035475 31975 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrXpU3x/build/release/bin/postgresql.jar:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrXpU3x/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:32:31.035497 31975 mini_ranger_kms.cc:240] Using host: 127.31.57.212
I20260809 19:32:31.039412 31975 mini_ranger_kms.cc:276] Ranger KMS PID: 19350
I20260809 19:32:31.039485 31975 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.57.212:57041
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:32:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 19:32:33 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303953, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:33.732853 31975 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.389s user 0.005s sys 0.173s
I20260809 19:32:33.732977 31975 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:32:33.733035 31975 mini_ranger_kms.cc:401] 127.31.57.212:57041/kms/v1/keys
Aug 09 19:32:33 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303953, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
Aug 09 19:32:34 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303953, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.57.212@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
Aug 09 19:32:36 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:32:36.272315 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41949
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41949
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:36.349177 19448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:36.349385 19448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:36.349418 19448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:36.350466 19448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:36.350526 19448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:36.350550 19448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:36.350576 19448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:36.350596 19448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:36.352074 19448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.57.212:57041/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41949
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:36.352325 19448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:36.352526 19448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:36.357118 19459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:36.357120 19458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:36.357234 19448 server_base.cc:1034] running on GCE node
W20260809 19:32:36.357165 19462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:36.357537 19448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:36.357797 19448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:36.358970 19448 hybrid_clock.cc:648] HybridClock initialized: now 1786303956358954 us; error 30 us; skew 500 ppm
Aug 09 19:32:36 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303956, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:36.365821 19448 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:36.366263 19448 webserver.cc:460] Webserver started at http://127.31.57.254:42447/ using document root <none> and password file <none>
I20260809 19:32:36.366447 19448 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:36.366482 19448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:36.366613 19448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:32:36 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303956, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:36.417328 19448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "448b48ce18944885b1ca1536784f21d5"
format_stamp: "Formatted at 2026-08-09 19:32:36 on dist-test-slave-x4qh"
server_key: "124f9edc2ce1013d7df13e2104b11b70"
server_key_iv: "6657e14115788a76bad36b815c9696b5"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:36.417845 19448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "448b48ce18944885b1ca1536784f21d5"
format_stamp: "Formatted at 2026-08-09 19:32:36 on dist-test-slave-x4qh"
server_key: "124f9edc2ce1013d7df13e2104b11b70"
server_key_iv: "6657e14115788a76bad36b815c9696b5"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:36.419792 19448 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 19:32:36.433866 19474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:36.434019 19448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:36.434108 19448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "448b48ce18944885b1ca1536784f21d5"
format_stamp: "Formatted at 2026-08-09 19:32:36 on dist-test-slave-x4qh"
server_key: "124f9edc2ce1013d7df13e2104b11b70"
server_key_iv: "6657e14115788a76bad36b815c9696b5"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:36.434190 19448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:36.452056 19448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:36.454025 19448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:36.454191 19448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:36.458142 19448 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41949
I20260809 19:32:36.458277 19554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41949 every 8 connection(s)
I20260809 19:32:36.458595 19448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:36.459293 19557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:36.461393 19557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Bootstrap starting.
I20260809 19:32:36.461920 19557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:36.462216 19557 log.cc:824] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:36.462896 19557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: No bootstrap required, opened a new log
I20260809 19:32:36.464882 19557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:36.465024 19557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:36.465047 19557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448b48ce18944885b1ca1536784f21d5, State: Initialized, Role: FOLLOWER
I20260809 19:32:36.465270 19557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:36.465364 19557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:36.465415 19557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:36.465464 19557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:36.465998 19557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:36.466110 19557 leader_election.cc:302] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 448b48ce18944885b1ca1536784f21d5; no voters:
I20260809 19:32:36.466297 19557 leader_election.cc:288] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:32:36.466332 19560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:36.466475 19560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 LEADER]: Becoming Leader. State: Replica: 448b48ce18944885b1ca1536784f21d5, State: Running, Role: LEADER
I20260809 19:32:36.466552 19557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:36.466627 19560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:36.466976 19561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "448b48ce18944885b1ca1536784f21d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } } }
I20260809 19:32:36.467070 19561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:36.466982 19562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 448b48ce18944885b1ca1536784f21d5. Latest consensus state: current_term: 1 leader_uuid: "448b48ce18944885b1ca1536784f21d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } } }
I20260809 19:32:36.467190 19562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:36.467304 19573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:36.467881 19573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:36.467979 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 19448
I20260809 19:32:36.468072 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:32:36.468216 19557 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:32:36.468230 31975 external_mini_cluster.cc:1428] Setting key 3865b4f606cb2b1757db140b2e9b315a
I20260809 19:32:36.472956 19573 catalog_manager.cc:1347] Generated new cluster ID: 6d1fb477956e44ddafa07babbc06959c
I20260809 19:32:36.473031 19573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:32:36 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:36.482807 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:36.470717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.31.57.254:41949)
0809 19:32:36.471131 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:36.471141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:36.471183 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:32:36.471439 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:36.471451 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:36.471553 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:36.471756 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:36.472812 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:36.474144 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:36.475234 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:36.475861 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:36.475962 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:36.478801 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:36.478805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:36.478805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:36.478816 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:36.480798 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:36.481198 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:36.481201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:36.481201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:36.481312 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:36.481523 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:36.481523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:36.481524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:36.481836 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:36.481959 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:36.482476 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 19:32:36.482598 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":170,"threads_started":1}
I20260809 19:32:36.485828 19573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:32:36.486461 19573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:36.490708 19573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Generated new TSK 0
I20260809 19:32:36.490872 19573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:32:37.097945 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:37.210458 19630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:37.210668 19630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:37.210702 19630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:37.211786 19630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:37.211843 19630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:37.211881 19630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:32:37.213282 19630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:37.213523 19630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:37.213769 19630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:37.218669 19640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.218760 19639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.218669 19646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:37.219666 19630 server_base.cc:1034] running on GCE node
I20260809 19:32:37.219872 19630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:37.220111 19630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:37.221259 19630 hybrid_clock.cc:648] HybridClock initialized: now 1786303957221238 us; error 36 us; skew 500 ppm
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:37.223824 19630 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:32:37.224385 19630 webserver.cc:460] Webserver started at http://127.31.57.193:36457/ using document root <none> and password file <none>
I20260809 19:32:37.224651 19630 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:37.224696 19630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:37.224828 19630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:37.237340 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "40be5365e6cd4f13b80248fed6fb86fd"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "5acb53088de81e0abfd7c2551a92a1a8"
server_key_iv: "2dd606c1225ea4c4fd7e0e5301ec8381"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.237773 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "40be5365e6cd4f13b80248fed6fb86fd"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "5acb53088de81e0abfd7c2551a92a1a8"
server_key_iv: "2dd606c1225ea4c4fd7e0e5301ec8381"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.243767 19630 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260809 19:32:37.254458 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.254690 19630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:37.254805 19630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "40be5365e6cd4f13b80248fed6fb86fd"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "5acb53088de81e0abfd7c2551a92a1a8"
server_key_iv: "2dd606c1225ea4c4fd7e0e5301ec8381"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.254892 19630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:37.272565 19630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:37.275853 19630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:37.276032 19630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:37.276294 19630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:37.276652 19630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:37.276739 19630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.276835 19630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:37.276908 19630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.284621 19630 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:42915
I20260809 19:32:37.285120 19630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:32:37.285719 19850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:42915 every 8 connection(s)
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:37.295126 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 19630
I20260809 19:32:37.295222 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:32:37.295464 31975 external_mini_cluster.cc:1428] Setting key 70e17922a7c2342095fde87f30b88b82
I20260809 19:32:37.297155 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.286349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.193:47503)
0809 19:32:37.286740 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:37.286786 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:37.287445 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:37.287842 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:37.287846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:37.288066 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:37.288364 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.288371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.289309 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.289311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.290549 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.290552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.291076 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.291077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.291224 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.291768 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:37.291777 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:37.293345 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:37.295371 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.295394 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.295396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.295712 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.295962 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.295963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.295963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.296102 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.296431 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:37.296438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:37.296596 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 19:32:37.296698 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 19:32:37.296858 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":154,"threads_started":1}
I20260809 19:32:37.297818 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.286377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:47503 (local address 127.31.57.254:41949)
0809 19:32:37.286678 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:37.286680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:37.287612 (+ 932us) server_negotiation.cc:407] Connection header received
0809 19:32:37.287647 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:37.287648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:37.287662 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:37.287752 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:37.288471 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.289206 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.290649 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.290945 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.291002 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.293444 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:37.293446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:37.293446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:37.293453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:37.295286 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.295795 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.295797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.295797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.295886 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.296198 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.296199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.296200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.296353 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:37.296421 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:37.297586 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0809 19:32:37.297682 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":85,"threads_started":1}
I20260809 19:32:37.298648 19851 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:37.298744 19851 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:37.298926 19851 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:37.299512 19503 ts_manager.cc:194] Registered new tserver with Master: 40be5365e6cd4f13b80248fed6fb86fd (127.31.57.193:42915)
I20260809 19:32:37.300374 19503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:47503
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:32:37.322327 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:37.431361 19862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:37.431641 19862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:37.431679 19862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:37.433349 19862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:37.433416 19862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:37.433462 19862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:32:37.435563 19862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:37.435791 19862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:37.436071 19862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 19:32:37.442759 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.442777 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.443300 19881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:37.443876 19862 server_base.cc:1034] running on GCE node
I20260809 19:32:37.444124 19862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:37.444428 19862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:37.445626 19862 hybrid_clock.cc:648] HybridClock initialized: now 1786303957445571 us; error 67 us; skew 500 ppm
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:37.449122 19862 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:32:37.449620 19862 webserver.cc:460] Webserver started at http://127.31.57.194:39155/ using document root <none> and password file <none>
I20260809 19:32:37.449906 19862 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:37.450012 19862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:37.450191 19862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:37.466620 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "f0c74f376fe24dd188de5b3a09092192"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "839d95d5e4337c4cc685d1a3d205436c"
server_key_iv: "843116b23f9afc978e3784a48f1aafde"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.467150 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "f0c74f376fe24dd188de5b3a09092192"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "839d95d5e4337c4cc685d1a3d205436c"
server_key_iv: "843116b23f9afc978e3784a48f1aafde"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.471836 19862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260809 19:32:37.501969 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.502252 19862 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.001s sys 0.001s
I20260809 19:32:37.502398 19862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "f0c74f376fe24dd188de5b3a09092192"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "839d95d5e4337c4cc685d1a3d205436c"
server_key_iv: "843116b23f9afc978e3784a48f1aafde"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.502518 19862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:37.526542 19862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:37.529206 19862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:37.529440 19862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:37.529811 19862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:37.530279 19862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:37.530375 19862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.530454 19862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:37.530514 19862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.539104 19862 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:37605
I20260809 19:32:37.539914 19862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:32:37.542496 20092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:37605 every 8 connection(s)
I20260809 19:32:37.543180 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 19862
I20260809 19:32:37.543290 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:32:37.543545 31975 external_mini_cluster.cc:1428] Setting key a9b7bfffce195666ecaffb89f82f6946
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:37.554044 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.543246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42665 (local address 127.31.57.254:41949)
0809 19:32:37.543795 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:37.543797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:37.543809 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:32:37.544414 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:37.544416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:37.544432 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:37.544470 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:37.545363 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.546064 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.547523 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.547806 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.547861 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.550518 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:37.550520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:37.550520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:37.550528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:37.552384 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.552810 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.552811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.552811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.552879 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.553204 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.553205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.553205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.553354 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:37.553404 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:37.553790 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 19:32:37.553896 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
I20260809 19:32:37.554852 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.543251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.194:42665)
0809 19:32:37.543485 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:37.543538 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:37.544246 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:37.544673 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:37.544679 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:37.544936 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:37.545238 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.545244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.546203 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.546204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.547408 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.547411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.547978 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.547980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.548131 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.548707 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:37.548717 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:37.550397 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:37.552455 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.552461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.552463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.552703 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.552964 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.552965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.552965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.553114 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.553460 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:37.553466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:37.553634 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 19:32:37.554410 (+ 776us) client_negotiation.cc:240] Negotiation successful
0809 19:32:37.554582 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260809 19:32:37.555599 20093 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:37.555691 20093 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:37.555881 20093 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:37.556255 19503 ts_manager.cc:194] Registered new tserver with Master: f0c74f376fe24dd188de5b3a09092192 (127.31.57.194:37605)
I20260809 19:32:37.556731 19503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:42665
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:32:37.584007 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:37.695156 20098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:37.695534 20098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:37.695628 20098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:37.697244 20098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:37.697373 20098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:37.697464 20098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:32:37.699584 20098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:37.700048 20098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:37.700366 20098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:37.708474 20105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.709508 20106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.709643 20109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:37.712904 20098 server_base.cc:1034] running on GCE node
I20260809 19:32:37.713239 20098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:37.713511 20098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:37.715587 20098 hybrid_clock.cc:648] HybridClock initialized: now 1786303957715571 us; error 48 us; skew 500 ppm
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:37.718874 20098 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:32:37.719357 20098 webserver.cc:460] Webserver started at http://127.31.57.195:45325/ using document root <none> and password file <none>
I20260809 19:32:37.719681 20098 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:37.719743 20098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:37.719892 20098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:37.739621 20098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "f361f2f18f504ae28802e90cc455b6b0"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "9354c28ca1e1bea9c1258ebfb7973173"
server_key_iv: "372521249c045d4ca25b2f4b03c9dd1d"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.740185 20098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "f361f2f18f504ae28802e90cc455b6b0"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "9354c28ca1e1bea9c1258ebfb7973173"
server_key_iv: "372521249c045d4ca25b2f4b03c9dd1d"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.742100 20098 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 19:32:37.753209 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.753613 20098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 19:32:37.753705 20098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "f361f2f18f504ae28802e90cc455b6b0"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "9354c28ca1e1bea9c1258ebfb7973173"
server_key_iv: "372521249c045d4ca25b2f4b03c9dd1d"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:37.753790 20098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:37.767978 20098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:37.770231 20098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:37.770375 20098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:37.770622 20098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:37.770999 20098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:37.771045 20098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.771073 20098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:37.771142 20098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:37.778861 20098 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:34937
I20260809 19:32:37.779330 20098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:32:37.787596 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 20098
I20260809 19:32:37.787684 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:32:37.787907 31975 external_mini_cluster.cc:1428] Setting key b97ee8a68bcb9483eb0fa4959dbd1b59
I20260809 19:32:37.791741 20307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:34937 every 8 connection(s)
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:37.811053 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.792684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:52309 (local address 127.31.57.254:41949)
0809 19:32:37.792742 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:37.792748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:37.795305 (+ 2557us) server_negotiation.cc:407] Connection header received
0809 19:32:37.795917 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:37.795918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:37.795930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:37.795992 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:37.796626 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.797113 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.798121 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.798423 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.798497 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.807809 (+ 9312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:37.807811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:37.807811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:37.807818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:37.809477 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.809971 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.809972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.809972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.810031 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:37.810338 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:37.810339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:37.810339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:37.810475 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:37.810520 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:37.810884 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 19:32:37.810966 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:32:37.813239 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:37.792393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.195:52309)
0809 19:32:37.795179 (+ 2786us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:37.795227 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:37.795797 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:37.796087 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:37.796091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:37.796276 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:37.796471 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.796475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.797209 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.797210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.797979 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:37.797981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:37.798582 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:37.798584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:37.798725 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:37.804985 (+ 6260us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:37.804996 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:37.807699 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:37.809577 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.809584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.809587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.809878 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.810107 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:37.810107 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:37.810108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:37.810263 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:37.810568 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:37.810574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:37.810728 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 19:32:37.811416 (+ 688us) client_negotiation.cc:240] Negotiation successful
0809 19:32:37.813002 (+ 1586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2727,"thread_start_us":115,"threads_started":1}
I20260809 19:32:37.813943 20308 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:37.814054 20308 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:37.814260 20308 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:37.814688 19503 ts_manager.cc:194] Registered new tserver with Master: f361f2f18f504ae28802e90cc455b6b0 (127.31.57.195:34937)
I20260809 19:32:37.815197 19503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:52309
I20260809 19:32:37.816787 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:32:37.817520 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 19630
I20260809 19:32:37.831929 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 19862
I20260809 19:32:37.839813 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 20098
I20260809 19:32:37.860488 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 19448
I20260809 19:32:37.871431 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41949
--webserver_interface=127.31.57.254
--webserver_port=42447
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41949
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:37.966120 20317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:37.966302 20317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:37.966322 20317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:37.967423 20317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:37.967460 20317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:37.967473 20317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:37.967521 20317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:37.967551 20317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:37.968964 20317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41949
--ranger_config_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=42447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:37.969180 20317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:37.969385 20317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:37.973707 20323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.973850 20324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:37.973868 20327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:37.974123 20317 server_base.cc:1034] running on GCE node
I20260809 19:32:37.974359 20317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:37.974581 20317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:37.975713 20317 hybrid_clock.cc:648] HybridClock initialized: now 1786303957975702 us; error 28 us; skew 500 ppm
Aug 09 19:32:37 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:37.977958 20317 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:37.978426 20317 webserver.cc:460] Webserver started at http://127.31.57.254:42447/ using document root <none> and password file <none>
I20260809 19:32:37.978636 20317 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:37.978685 20317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303957, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:38.055299 20317 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:38.056064 20337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.056273 20317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:38.056375 20317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "448b48ce18944885b1ca1536784f21d5"
format_stamp: "Formatted at 2026-08-09 19:32:36 on dist-test-slave-x4qh"
server_key: "124f9edc2ce1013d7df13e2104b11b70"
server_key_iv: "6657e14115788a76bad36b815c9696b5"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:38.056741 20317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:38.072023 20317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:38.073508 20317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:38.073673 20317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:38.077965 20317 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41949
I20260809 19:32:38.077997 20414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41949 every 8 connection(s)
I20260809 19:32:38.078368 20317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:38.078816 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 20317
I20260809 19:32:38.079123 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:42915
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:41949
--webserver_port=36457
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:38.080220 20417 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:32:38.081231 20417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Bootstrap starting.
I20260809 19:32:38.089026 20417 log.cc:824] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:38.090619 20417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:32:38.090905 20417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Bootstrap complete.
I20260809 19:32:38.093101 20417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:38.093307 20417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448b48ce18944885b1ca1536784f21d5, State: Initialized, Role: FOLLOWER
I20260809 19:32:38.093456 20417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:38.093525 20417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:38.093572 20417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:38.093618 20417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:32:38.094587 20417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:38.094672 20417 leader_election.cc:302] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 448b48ce18944885b1ca1536784f21d5; no voters:
I20260809 19:32:38.094887 20417 leader_election.cc:288] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:32:38.094918 20423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:32:38.095037 20423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [term 2 LEADER]: Becoming Leader. State: Replica: 448b48ce18944885b1ca1536784f21d5, State: Running, Role: LEADER
I20260809 19:32:38.095145 20417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:38.095126 20423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } }
I20260809 19:32:38.095430 20423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 448b48ce18944885b1ca1536784f21d5. Latest consensus state: current_term: 2 leader_uuid: "448b48ce18944885b1ca1536784f21d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } } }
I20260809 19:32:38.095497 20423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:38.095690 20424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "448b48ce18944885b1ca1536784f21d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448b48ce18944885b1ca1536784f21d5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41949 } } }
I20260809 19:32:38.095767 20424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:38.095852 20428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:38.096478 20428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:38.096594 20428 catalog_manager.cc:1259] Loaded cluster ID: 6d1fb477956e44ddafa07babbc06959c
I20260809 19:32:38.096625 20428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:32:38.096932 20417 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:32:38.100658 20428 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:38.101279 20428 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Loaded TSK: 0
I20260809 19:32:38.101577 20428 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:32:38.173548 20419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:38.173811 20419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:38.173851 20419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:38.175763 20419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:38.175843 20419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:38.175899 20419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:32:38.178215 20419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:42915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=36457
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:38.178431 20419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:38.178669 20419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:38.184859 20491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:38.184871 20496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:38.185233 20493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:38.185834 20419 server_base.cc:1034] running on GCE node
I20260809 19:32:38.186019 20419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:38.186280 20419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:38.187443 20419 hybrid_clock.cc:648] HybridClock initialized: now 1786303958187413 us; error 43 us; skew 500 ppm
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:38.190583 20419 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:32:38.191090 20419 webserver.cc:460] Webserver started at http://127.31.57.193:36457/ using document root <none> and password file <none>
I20260809 19:32:38.191354 20419 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:38.191449 20419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:38.212452 20419 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:38.214162 20512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.214335 20419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 19:32:38.214421 20419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "40be5365e6cd4f13b80248fed6fb86fd"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "5acb53088de81e0abfd7c2551a92a1a8"
server_key_iv: "2dd606c1225ea4c4fd7e0e5301ec8381"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:38.214862 20419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:38.228835 20419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:38.231081 20419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:38.231248 20419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:38.231474 20419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:38.231809 20419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:38.231855 20419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.231894 20419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:38.231920 20419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.240129 20419 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:42915
I20260809 19:32:38.240252 20717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:42915 every 8 connection(s)
I20260809 19:32:38.240578 20419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:32:38.245499 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 20419
I20260809 19:32:38.245767 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:37605
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:41949
--webserver_port=39155
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:38.256788 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.241807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.193:34639)
0809 19:32:38.242042 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:38.242080 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:38.242584 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:38.243344 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:38.243348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:38.243564 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:38.243811 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.243817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.247815 (+ 3998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.247816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.248953 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.248956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.249837 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.249838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.249959 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.250476 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:38.250485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:38.251993 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:38.254210 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.254217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.254219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.254514 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.255445 (+ 931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.255446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.255446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.255579 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.256188 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:38.256194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:38.256329 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 19:32:38.256431 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 19:32:38.256575 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260809 19:32:38.257310 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.241876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:34639 (local address 127.31.57.254:41949)
0809 19:32:38.242256 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:38.242263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:38.242305 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:32:38.243092 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:38.243099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:38.243140 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:38.243287 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:38.243945 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.247697 (+ 3752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.249052 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.249628 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.249729 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.252122 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:38.252126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:38.252126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:38.252137 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:38.254084 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.254619 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.254623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.254624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.254743 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.255719 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.255720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.255720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.256009 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:38.256166 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:38.257000 (+ 834us) server_negotiation.cc:299] Negotiation successful
0809 19:32:38.257134 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":65,"threads_started":1}
I20260809 19:32:38.258179 20719 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:38.258276 20719 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.258451 20719 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.259387 20719 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41949 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:32:38.259461 20719 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.259562 20719 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:38.260015 20719 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.260092 20719 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.260458 20719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:32:38.352473 20729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:38.352710 20729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:38.352739 20729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:38.354342 20729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:38.354405 20729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:38.354447 20729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:32:38.356448 20729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:37605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=39155
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:38.356658 20729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:38.356909 20729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:38.364322 20737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:38.365211 20729 server_base.cc:1034] running on GCE node
W20260809 19:32:38.365275 20742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:38.365509 20739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:38.365852 20729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:38.366103 20729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:38.367254 20729 hybrid_clock.cc:648] HybridClock initialized: now 1786303958367245 us; error 32 us; skew 500 ppm
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:38.369845 20729 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:32:38.370350 20729 webserver.cc:460] Webserver started at http://127.31.57.194:39155/ using document root <none> and password file <none>
I20260809 19:32:38.370577 20729 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:38.370644 20729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:38.388602 20729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:38.389742 20753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.390008 20729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:38.390108 20729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "f0c74f376fe24dd188de5b3a09092192"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "839d95d5e4337c4cc685d1a3d205436c"
server_key_iv: "843116b23f9afc978e3784a48f1aafde"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:38.390551 20729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:38.411816 20729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:38.413852 20729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:38.414022 20729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:38.414333 20729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:38.414721 20729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:38.414777 20729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.414811 20729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:38.414840 20729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.423362 20729 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:37605
I20260809 19:32:38.423873 20729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:32:38.424405 20947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:37605 every 8 connection(s)
I20260809 19:32:38.425882 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 20729
I20260809 19:32:38.426193 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.57.212:57041/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:34937
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:41949
--webserver_port=45325
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:38.438913 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.425365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:38769 (local address 127.31.57.254:41949)
0809 19:32:38.425424 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:38.425426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:38.425437 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:32:38.426018 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:38.426020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:38.426036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:38.426104 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:38.426771 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.427480 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.428660 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.428897 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.428968 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.435796 (+ 6828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:38.435798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:38.435798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:38.435805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:38.437437 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.437899 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.437900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.437901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.437959 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.438210 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.438211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.438212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.438323 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:38.438390 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:38.438724 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 19:32:38.438821 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:32:38.439415 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.425062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.194:38769)
0809 19:32:38.425259 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:38.425306 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:38.425906 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:38.426175 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:38.426179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:38.426392 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:38.426650 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.426657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.427582 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.427583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.428569 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.428572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.429009 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.429010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.429129 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.429639 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:38.429647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:38.435666 (+ 6019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:38.437532 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.437538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.437540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.437836 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.438031 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.438032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.438032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.438136 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.438395 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:38.438401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:38.438562 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 19:32:38.439065 (+ 503us) client_negotiation.cc:240] Negotiation successful
0809 19:32:38.439211 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260809 19:32:38.439826 20948 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:38.439939 20948 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.440161 20948 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.440804 20948 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41949 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:32:38.440887 20948 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.440963 20948 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:38.441293 20948 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.441383 20948 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.441653 20948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:32:38.545470 20952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:38.545702 20952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:38.545737 20952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:38.547214 20952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:38.547271 20952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:38.547309 20952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:32:38.549233 20952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.57.212:57041/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:34937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=45325
--tserver_master_addrs=127.31.57.254:41949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:38.549463 20952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:38.549772 20952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:38.555986 20967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:38.556236 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:38.556365 20952 server_base.cc:1034] running on GCE node
W20260809 19:32:38.556468 20965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:38.556679 20952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:38.556953 20952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:38.558099 20952 hybrid_clock.cc:648] HybridClock initialized: now 1786303958558081 us; error 35 us; skew 500 ppm
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:38.560702 20952 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:32:38.561141 20952 webserver.cc:460] Webserver started at http://127.31.57.195:45325/ using document root <none> and password file <none>
I20260809 19:32:38.561370 20952 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:38.561450 20952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:38.580768 20952 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 19:32:38.581842 20980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.582024 20952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:32:38.582145 20952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "f361f2f18f504ae28802e90cc455b6b0"
format_stamp: "Formatted at 2026-08-09 19:32:37 on dist-test-slave-x4qh"
server_key: "9354c28ca1e1bea9c1258ebfb7973173"
server_key_iv: "372521249c045d4ca25b2f4b03c9dd1d"
server_key_version: "kuduclusterkey@0"
I20260809 19:32:38.582495 20952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:38.598268 20952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:38.600555 20952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:38.600746 20952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:38.601015 20952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:38.601438 20952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:38.601549 20952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.601657 20952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:38.601696 20952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:38.609632 20952 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:34937
I20260809 19:32:38.610004 20952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:32:38.610634 21154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:34937 every 8 connection(s)
I20260809 19:32:38.615831 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 20952
Aug 09 19:32:38 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303958, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:38.620481 20728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.611032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:55759 (local address 127.31.57.254:41949)
0809 19:32:38.611077 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:38.611079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:38.611101 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:32:38.611641 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:38.611643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:38.611658 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:38.611737 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:38.612264 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.612799 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.614095 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.614325 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.614424 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.616997 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:38.616998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:38.616998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:38.617006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:38.618826 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.619402 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.619403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.619404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.619468 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.619753 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.619753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.619754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.619925 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:38.619988 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:38.620302 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 19:32:38.620389 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:32:38.620632 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.610749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:41949 (local address 127.31.57.195:55759)
0809 19:32:38.610897 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:38.610936 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:38.611528 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:38.611805 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:38.611808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:38.611981 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:38.612164 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.612168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.612934 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.612935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.613970 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:38.613974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.614455 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.614457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:38.614612 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.615169 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:38.615179 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:38.616887 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:38.618931 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.618937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.618940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.619266 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.619549 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:38.619550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:38.619550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:38.619676 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:38.620017 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:38.620031 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:38.620148 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 19:32:38.620263 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:32:38.620403 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260809 19:32:38.621724 21155 heartbeater.cc:347] Connected to a master server at 127.31.57.254:41949
I20260809 19:32:38.621814 21155 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.622009 21155 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.622926 21155 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:41949 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:32:38.623013 21155 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.623136 21155 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:38.623541 21155 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:38.623670 21155 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
W20260809 19:32:38.623996 21155 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:32:38.629073 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:38.616269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.31.57.254:41949)
0809 19:32:38.616387 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:38.616389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:38.616536 (+ 147us) server_negotiation.cc:407] Connection header received
0809 19:32:38.616649 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:38.616650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:38.616663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:38.616715 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:38.617892 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.618641 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.620878 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:38.621202 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:38.621276 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:38.623901 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:38.623903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:38.623903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:38.623911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:38.625552 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.627149 (+ 1597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.627150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.627151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.627208 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:38.627458 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:38.627458 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:38.627459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:38.627568 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:38.628898 (+ 1330us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:38.628938 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:32:38.629007 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
Aug 09 19:32:39 dist-test-slave-x4qh krb5kdc[18720](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303959, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:39.260618 20719 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:39.260754 20719 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:39.261138 20361 ts_manager.cc:194] Registered new tserver with Master: 40be5365e6cd4f13b80248fed6fb86fd (127.31.57.193:42915)
I20260809 19:32:39.262022 20361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:34639
I20260809 19:32:39.441844 20948 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:39.441987 20948 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:39.442301 20361 ts_manager.cc:194] Registered new tserver with Master: f0c74f376fe24dd188de5b3a09092192 (127.31.57.194:37605)
I20260809 19:32:39.442842 20361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:38769
Aug 09 19:32:39 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303959, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
I20260809 19:32:39.624202 21155 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:39.624380 21155 heartbeater.cc:511] Master 127.31.57.254:41949 requested a full tablet report, sending...
I20260809 19:32:39.624737 20361 ts_manager.cc:194] Registered new tserver with Master: f361f2f18f504ae28802e90cc455b6b0 (127.31.57.195:34937)
I20260809 19:32:39.625286 20361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:55759
I20260809 19:32:39.626605 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:32:39.639735 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.627637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.31.57.254:41949)
0809 19:32:39.627772 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.627774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.631636 (+ 3862us) server_negotiation.cc:407] Connection header received
0809 19:32:39.631669 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.631670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.631682 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.631752 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:39.632430 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.633218 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.634306 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.634610 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.634687 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.636237 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:39.636239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:39.636239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:39.636246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:39.638196 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:39.638674 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:39.638675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:39.638675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:39.638742 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:39.639058 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:39.639059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:39.639059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:39.639234 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:39.639339 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.639550 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.639638 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260809 19:32:39.642184 20361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:32:39.644483 20361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:32:39.656141 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:34937 (local address 127.0.0.1:53210)
0809 19:32:39.651567 (+ 3810us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.651576 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.651635 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.652134 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:39.652135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:39.652137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:39.652251 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.652253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.653437 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.653439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.654724 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.654728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.655831 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.655832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.655919 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.655933 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:32:39.656004 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:32:39.656061 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3793,"thread_start_us":78,"threads_started":1}
I20260809 19:32:39.656560 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.31.57.195:34937)
0809 19:32:39.651507 (+ 3659us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.651511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.651690 (+ 179us) server_negotiation.cc:407] Connection header received
0809 19:32:39.651726 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.651728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.651742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.651807 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:39.652341 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.653311 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.654831 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.655652 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.655730 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.655793 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.656442 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.656484 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3630,"thread_start_us":51,"threads_started":1}
I20260809 19:32:39.657467 21058 tablet_service.cc:1467] Processing CreateTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 (DEFAULT_TABLE table=test-table [id=3e4c2262782f44a7858179f4b8d68aa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:39.657845 21058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323b3c422f51411aa3ef0c7d5cdb8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:39.658116 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42915 (local address 127.0.0.1:56802)
0809 19:32:39.648165 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.648216 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.648270 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.652434 (+ 4164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:39.652435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:39.652437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:39.652488 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.652489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.656078 (+ 3589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.656078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.657071 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.657073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.657915 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.657916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.657977 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.657981 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:32:39.658021 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:32:39.658054 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":77,"threads_started":1}
I20260809 19:32:39.659541 21193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37605 (local address 127.0.0.1:49254)
0809 19:32:39.651466 (+ 3958us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.651486 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.651541 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.651799 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:39.651802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:39.651804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:39.651856 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.651858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.654597 (+ 2739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.654598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.655971 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.655973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.659305 (+ 3332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.659306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.659360 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.659363 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:32:39.659419 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:32:39.659453 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3940,"mutex_wait_us":10,"thread_start_us":91,"threads_started":1}
I20260809 19:32:39.659952 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.31.57.193:42915)
0809 19:32:39.651929 (+ 4176us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.651933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.651940 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:39.651970 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.651972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.651984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.652052 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:39.652566 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.653578 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.657177 (+ 3599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.657771 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.657838 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.657884 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.659844 (+ 1960us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.659885 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4146,"thread_start_us":54,"threads_started":1}
I20260809 19:32:39.660234 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.647981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49254 (local address 127.31.57.194:37605)
0809 19:32:39.648118 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.648122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.651601 (+ 3479us) server_negotiation.cc:407] Connection header received
0809 19:32:39.651638 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.651640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.651654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.651717 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:39.651924 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.654490 (+ 2566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.658258 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.659152 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.659231 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.659274 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.660117 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.660168 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":47,"threads_started":1}
I20260809 19:32:39.660997 20599 tablet_service.cc:1467] Processing CreateTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 (DEFAULT_TABLE table=test-table [id=3e4c2262782f44a7858179f4b8d68aa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:39.661324 20599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323b3c422f51411aa3ef0c7d5cdb8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:39.664772 21207 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Bootstrap starting.
I20260809 19:32:39.665364 20845 tablet_service.cc:1467] Processing CreateTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 (DEFAULT_TABLE table=test-table [id=3e4c2262782f44a7858179f4b8d68aa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:39.665683 21207 tablet_bootstrap.cc:654] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:39.665725 20845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323b3c422f51411aa3ef0c7d5cdb8642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:39.666042 21207 log.cc:824] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:39.666499 21206 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Bootstrap starting.
I20260809 19:32:39.666954 21207 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: No bootstrap required, opened a new log
I20260809 19:32:39.667018 21207 ts_tablet_manager.cc:1397] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:32:39.667099 21206 tablet_bootstrap.cc:654] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:39.667413 21206 log.cc:824] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:39.669050 21213 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Bootstrap starting.
I20260809 19:32:39.669754 21213 tablet_bootstrap.cc:654] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:39.669740 21207 raft_consensus.cc:348] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.669934 21207 raft_consensus.cc:374] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:39.670003 21207 raft_consensus.cc:729] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40be5365e6cd4f13b80248fed6fb86fd, State: Initialized, Role: FOLLOWER
I20260809 19:32:39.670118 21213 log.cc:824] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:39.670172 21207 consensus_queue.cc:260] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.670461 21207 ts_tablet_manager.cc:1428] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:32:39.670600 20719 heartbeater.cc:503] Master 127.31.57.254:41949 was elected leader, sending a full tablet report...
I20260809 19:32:39.670969 21206 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: No bootstrap required, opened a new log
I20260809 19:32:39.671041 21206 ts_tablet_manager.cc:1397] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 19:32:39.673547 21206 raft_consensus.cc:348] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.673722 21206 raft_consensus.cc:374] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:39.673751 21206 raft_consensus.cc:729] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f361f2f18f504ae28802e90cc455b6b0, State: Initialized, Role: FOLLOWER
I20260809 19:32:39.673856 21206 consensus_queue.cc:260] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.674156 21206 ts_tablet_manager.cc:1428] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:32:39.674257 21155 heartbeater.cc:503] Master 127.31.57.254:41949 was elected leader, sending a full tablet report...
I20260809 19:32:39.675100 21213 tablet_bootstrap.cc:492] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: No bootstrap required, opened a new log
I20260809 19:32:39.675184 21213 ts_tablet_manager.cc:1397] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 19:32:39.677523 21213 raft_consensus.cc:348] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.677672 21213 raft_consensus.cc:374] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:39.677701 21213 raft_consensus.cc:729] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c74f376fe24dd188de5b3a09092192, State: Initialized, Role: FOLLOWER
I20260809 19:32:39.677801 21213 consensus_queue.cc:260] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.678004 21213 ts_tablet_manager.cc:1428] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:32:39.678058 20948 heartbeater.cc:503] Master 127.31.57.254:41949 was elected leader, sending a full tablet report...
Aug 09 19:32:39 dist-test-slave-x4qh krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303959, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for HTTP/127.31.57.212@KRBTEST.COM
W20260809 19:32:39.742498 20726 tablet.cc:2369] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:32:39.747128 21216 raft_consensus.cc:482] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:39.747267 21216 raft_consensus.cc:504] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.747675 21216 leader_election.cc:288] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40be5365e6cd4f13b80248fed6fb86fd (127.31.57.193:42915), f0c74f376fe24dd188de5b3a09092192 (127.31.57.194:37605)
I20260809 19:32:39.753834 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.748449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:41877 (local address 127.31.57.194:37605)
0809 19:32:39.749135 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.749137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.749145 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:32:39.749173 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.749174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.749183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.749230 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:39.750808 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.751536 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.752909 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.753426 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.753474 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.753483 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.753717 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.753758 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659}
I20260809 19:32:39.754120 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.748382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:56325 (local address 127.31.57.193:42915)
0809 19:32:39.749290 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.749292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.749300 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:32:39.749324 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.749325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.749334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.749380 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:39.751217 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.751905 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.753121 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.753727 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.753779 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.753786 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.754015 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.754053 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883}
I20260809 19:32:39.754236 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.748089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37605 (local address 127.31.57.195:41877)
0809 19:32:39.748993 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.749002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.749040 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.750607 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:39.750608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:39.750611 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:39.750693 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.750695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.751630 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.751630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.752810 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.752813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.753567 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.753568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.753636 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.753639 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:32:39.754102 (+ 463us) client_negotiation.cc:240] Negotiation successful
0809 19:32:39.754144 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":77,"threads_started":1}
I20260809 19:32:39.754594 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.747806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42915 (local address 127.31.57.195:56325)
0809 19:32:39.748826 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.748838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.748897 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.750515 (+ 1618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:39.750516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:39.750532 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:39.751096 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.751099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.752005 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.752006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.752998 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:39.753002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.753847 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.753848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:39.753936 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.753939 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:32:39.754473 (+ 534us) client_negotiation.cc:240] Negotiation successful
0809 19:32:39.754515 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":114,"threads_started":1}
I20260809 19:32:39.754557 20865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323b3c422f51411aa3ef0c7d5cdb8642" candidate_uuid: "f361f2f18f504ae28802e90cc455b6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0c74f376fe24dd188de5b3a09092192" is_pre_election: true
I20260809 19:32:39.754701 20865 raft_consensus.cc:2454] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f361f2f18f504ae28802e90cc455b6b0 in term 0.
I20260809 19:32:39.754971 20984 leader_election.cc:302] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0c74f376fe24dd188de5b3a09092192, f361f2f18f504ae28802e90cc455b6b0; no voters:
I20260809 19:32:39.755065 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323b3c422f51411aa3ef0c7d5cdb8642" candidate_uuid: "f361f2f18f504ae28802e90cc455b6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40be5365e6cd4f13b80248fed6fb86fd" is_pre_election: true
I20260809 19:32:39.755184 21216 raft_consensus.cc:2781] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:32:39.755218 20626 raft_consensus.cc:2454] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f361f2f18f504ae28802e90cc455b6b0 in term 0.
I20260809 19:32:39.755242 21216 raft_consensus.cc:482] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:32:39.755266 21216 raft_consensus.cc:3037] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:39.756199 21216 raft_consensus.cc:504] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.756335 21216 leader_election.cc:288] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [CANDIDATE]: Term 1 election: Requested vote from peers 40be5365e6cd4f13b80248fed6fb86fd (127.31.57.193:42915), f0c74f376fe24dd188de5b3a09092192 (127.31.57.194:37605)
I20260809 19:32:39.756523 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323b3c422f51411aa3ef0c7d5cdb8642" candidate_uuid: "f361f2f18f504ae28802e90cc455b6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40be5365e6cd4f13b80248fed6fb86fd"
I20260809 19:32:39.756577 20865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323b3c422f51411aa3ef0c7d5cdb8642" candidate_uuid: "f361f2f18f504ae28802e90cc455b6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0c74f376fe24dd188de5b3a09092192"
I20260809 19:32:39.756600 20626 raft_consensus.cc:3037] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:39.756647 20865 raft_consensus.cc:3037] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:39.757498 20865 raft_consensus.cc:2454] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f361f2f18f504ae28802e90cc455b6b0 in term 1.
I20260809 19:32:39.757503 20626 raft_consensus.cc:2454] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f361f2f18f504ae28802e90cc455b6b0 in term 1.
I20260809 19:32:39.757647 20984 leader_election.cc:302] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0c74f376fe24dd188de5b3a09092192, f361f2f18f504ae28802e90cc455b6b0; no voters:
I20260809 19:32:39.757716 21216 raft_consensus.cc:2781] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:39.757853 21216 raft_consensus.cc:686] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 1 LEADER]: Becoming Leader. State: Replica: f361f2f18f504ae28802e90cc455b6b0, State: Running, Role: LEADER
I20260809 19:32:39.757956 21216 consensus_queue.cc:237] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } }
I20260809 19:32:39.758671 20361 catalog_manager.cc:5495] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f361f2f18f504ae28802e90cc455b6b0 (127.31.57.195). New cstate: current_term: 1 leader_uuid: "f361f2f18f504ae28802e90cc455b6b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f361f2f18f504ae28802e90cc455b6b0" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 34937 } health_report { overall_health: HEALTHY } } }
I20260809 19:32:39.766224 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.762785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.31.57.195:34937)
0809 19:32:39.762828 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:39.762830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:39.762839 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:32:39.762868 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:39.762869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:39.762884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:39.762957 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:32:39.763111 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.763810 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.765322 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:39.765603 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:39.765670 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:39.765895 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:32:39.766004 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:32:39.766100 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:39.766144 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:32:39.766168 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:32:39.768007 20865 raft_consensus.cc:1261] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 1 FOLLOWER]: Refusing update from remote peer f361f2f18f504ae28802e90cc455b6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:39.768297 21216 consensus_queue.cc:1035] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c74f376fe24dd188de5b3a09092192" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 37605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:39.768430 20626 raft_consensus.cc:1261] T 323b3c422f51411aa3ef0c7d5cdb8642 P 40be5365e6cd4f13b80248fed6fb86fd [term 1 FOLLOWER]: Refusing update from remote peer f361f2f18f504ae28802e90cc455b6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:39.768714 21216 consensus_queue.cc:1035] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40be5365e6cd4f13b80248fed6fb86fd" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 42915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:39.772248 21225 mvcc.cc:205] Tried to move back new op lower bound from 7316701019207290880 to 7316701019169062912. Current Snapshot: MvccSnapshot[applied={T|T < 7316701019207290880}]
I20260809 19:32:39.772802 21231 mvcc.cc:205] Tried to move back new op lower bound from 7316701019207290880 to 7316701019169062912. Current Snapshot: MvccSnapshot[applied={T|T < 7316701019207290880}]
I20260809 19:32:39.777581 20359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:39.777660 20359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:39.778344 20359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 448b48ce18944885b1ca1536784f21d5: Sending DeleteTablet for 3 replicas of tablet 323b3c422f51411aa3ef0c7d5cdb8642
I20260809 19:32:39.778688 21226 mvcc.cc:205] Tried to move back new op lower bound from 7316701019207290880 to 7316701019169062912. Current Snapshot: MvccSnapshot[applied={T|T < 7316701019207290880}]
I20260809 19:32:39.778877 20599 tablet_service.cc:1514] Processing DeleteTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:39 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:56802
I20260809 19:32:39.778996 20845 tablet_service.cc:1514] Processing DeleteTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:39 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:49254
I20260809 19:32:39.779057 21058 tablet_service.cc:1514] Processing DeleteTablet for tablet 323b3c422f51411aa3ef0c7d5cdb8642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:39 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53210
I20260809 19:32:39.779243 21245 tablet_replica.cc:330] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: stopping tablet replica
I20260809 19:32:39.779345 21243 tablet_replica.cc:330] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: stopping tablet replica
I20260809 19:32:39.779452 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 20419
I20260809 19:32:39.779564 21245 raft_consensus.cc:2229] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:32:39.779698 21245 raft_consensus.cc:2258] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:39.789659 21245 ts_tablet_manager.cc:1905] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:39.794320 21245 ts_tablet_manager.cc:1918] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:39.794433 21245 log.cc:1196] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/323b3c422f51411aa3ef0c7d5cdb8642
I20260809 19:32:39.794740 21245 ts_tablet_manager.cc:1939] T 323b3c422f51411aa3ef0c7d5cdb8642 P f361f2f18f504ae28802e90cc455b6b0: Deleting consensus metadata
I20260809 19:32:39.795239 20339 catalog_manager.cc:4841] TS f361f2f18f504ae28802e90cc455b6b0 (127.31.57.195:34937): tablet 323b3c422f51411aa3ef0c7d5cdb8642 (table test-table [id=3e4c2262782f44a7858179f4b8d68aa0]) successfully deleted
I20260809 19:32:39.817322 21243 raft_consensus.cc:2229] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:39.817639 21243 raft_consensus.cc:2258] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:39.819130 21243 ts_tablet_manager.cc:1905] T 323b3c422f51411aa3ef0c7d5cdb8642 P f0c74f376fe24dd188de5b3a09092192: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 19:32:39.823961 20340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:42915 (error 108)
I20260809 19:32:39.824887 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 20729
I20260809 19:32:39.825080 21193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:39.824938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:42915 (local address 127.0.0.1:56810)
0809 19:32:39.824989 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.825039 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:42915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:32:39.825487 20340 catalog_manager.cc:4584] TS 40be5365e6cd4f13b80248fed6fb86fd (127.31.57.193:42915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 323b3c422f51411aa3ef0c7d5cdb8642: Network error: Client connection negotiation failed: client connection to 127.31.57.193:42915: connect: Connection refused (error 111)
W20260809 19:32:39.839781 21193 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:32:39.839580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:37605 (local address 127.0.0.1:49258)
0809 19:32:39.839620 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:39.839632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:39.839681 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:39.839729 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.194:37605: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 19:32:39.839989 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 20952
I20260809 19:32:39.847488 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 20317
I20260809 19:32:39.862324 31975 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:32:40.212816 31975 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:32:40.212903 31975 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:32:40.568374 31975 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:32:40.568 UTC [18775] LOG: received smart shutdown request
2026-08-09 19:32:40.573 UTC [18775] LOG: background worker "logical replication launcher" (PID 18789) exited with exit code 1
2026-08-09 19:32:40.573 UTC [18782] LOG: shutting down
2026-08-09 19:32:40.713 UTC [18775] LOG: database system is shut down
2026-08-09T19:32:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44959 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:32:40 dist-test-slave-x4qh krb5kdc[21260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:32:40 dist-test-slave-x4qh krb5kdc[21260](info): set up 2 sockets
Aug 09 19:32:40 dist-test-slave-x4qh krb5kdc[21260](info): commencing operation
krb5kdc: starting...
W20260809 19:32:41.993675 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:32:42.104180 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46767
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:42.179807 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:42.180011 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:42.180047 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:42.181092 21284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:42.181152 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:42.181175 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:42.181197 21284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:42.181223 21284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:42.182610 21284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:42.182881 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:42.183099 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:42.187651 21298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:42.187651 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:42.187731 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:42.187824 21284 server_base.cc:1034] running on GCE node
I20260809 19:32:42.188180 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:42.188462 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:42.189637 21284 hybrid_clock.cc:648] HybridClock initialized: now 1786303962189601 us; error 48 us; skew 500 ppm
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:42.196086 21284 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:42.196480 21284 webserver.cc:460] Webserver started at http://127.31.57.254:34309/ using document root <none> and password file <none>
I20260809 19:32:42.196661 21284 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:42.196707 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:42.196794 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:42.197819 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "01a737a82ddf4311851f916b6f517949"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "516619c6d4bb1428cb4aa70934b362d7"
server_key_iv: "4a997923a3fb9006a19ef93f528029f6"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.198211 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "01a737a82ddf4311851f916b6f517949"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "516619c6d4bb1428cb4aa70934b362d7"
server_key_iv: "4a997923a3fb9006a19ef93f528029f6"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.199618 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:32:42.200352 21310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:42.200526 21284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:42.200634 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "01a737a82ddf4311851f916b6f517949"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "516619c6d4bb1428cb4aa70934b362d7"
server_key_iv: "4a997923a3fb9006a19ef93f528029f6"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.200721 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:42.234010 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:42.235867 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:42.236037 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:42.240091 21284 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:46767
I20260809 19:32:42.240120 21394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:46767 every 8 connection(s)
I20260809 19:32:42.240448 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:42.241261 21395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:42.242597 21395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:42.248634 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 21284
I20260809 19:32:42.248790 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:32:42.249044 31975 external_mini_cluster.cc:1428] Setting key 7b4c33ecfe913e02e1608d231e9948fd
I20260809 19:32:42.251782 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.243212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46767 (local address 127.0.0.1:45946)
0809 19:32:42.243427 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.243466 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.243919 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.244189 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.244191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.244464 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.244603 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.244607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.245588 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.245588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.246223 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.246224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.246781 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.246783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.246848 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.247333 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.247341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.248642 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.250410 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.250417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.250418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.250607 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.250856 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.250857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.250857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.250947 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.251294 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.251299 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.251354 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.251454 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.251635 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":67,"spinlock_wait_cycles":6912,"thread_start_us":104,"threads_started":1}
I20260809 19:32:42.251797 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.243229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.31.57.254:46767)
0809 19:32:42.243435 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.243443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.243562 (+ 119us) server_negotiation.cc:407] Connection header received
0809 19:32:42.244064 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.244066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.244084 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.244183 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.244705 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.245476 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.246365 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.246684 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.246813 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.248753 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.248759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.248760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.248766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.250312 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.250690 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.250693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.250693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.250780 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.251028 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.251029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.251029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.251220 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.251306 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.251475 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.251580 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260809 19:32:42.252770 21395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:32:42.253098 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.252985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49610)
0809 19:32:42.253030 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.253069 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:32:42.253228 21313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
I20260809 19:32:42.253674 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.253585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49626)
0809 19:32:42.253631 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.253653 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:32:42.253793 21395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:36477: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.57.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
I20260809 19:32:42.274065 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:36477
--webserver_interface=127.31.57.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:42.303002 21395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } attempt: 1
I20260809 19:32:42.303508 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.303353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49630)
0809 19:32:42.303435 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.303477 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 19:32:42.303884 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.303770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49644)
0809 19:32:42.303813 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.303839 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:32:42.303989 21395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:36477: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
W20260809 19:32:42.354009 21406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:42.354184 21406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:42.354204 21406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:42.355325 21406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:42.355397 21406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:42.355412 21406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:42.355425 21406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:42.355476 21406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:42.356866 21406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:36477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.57.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:42.357051 21406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:42.357242 21406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:42.361610 21426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:42.361792 21419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:42.361833 21406 server_base.cc:1034] running on GCE node
W20260809 19:32:42.361991 21418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:42.362272 21406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:42.362540 21406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:42.363695 21406 hybrid_clock.cc:648] HybridClock initialized: now 1786303962363683 us; error 25 us; skew 500 ppm
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:42.366225 21406 init.cc:377] Logged in from keytab as kudu/127.31.57.253@KRBTEST.COM (short username kudu)
I20260809 19:32:42.366680 21406 webserver.cc:460] Webserver started at http://127.31.57.253:45381/ using document root <none> and password file <none>
I20260809 19:32:42.366902 21406 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:42.366942 21406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:42.367091 21406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:42.368093 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data/instance:
uuid: "e0fd6b0af1df44249a19a4a5a8a7e333"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "34c8fe81d3d490a48ab6594dd94e316f"
server_key_iv: "5f305311204d5bd822453494c37a1d9e"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.368474 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal/instance:
uuid: "e0fd6b0af1df44249a19a4a5a8a7e333"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "34c8fe81d3d490a48ab6594dd94e316f"
server_key_iv: "5f305311204d5bd822453494c37a1d9e"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.369760 21406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:32:42.370464 21437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:42.370755 21406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:42.370844 21406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "e0fd6b0af1df44249a19a4a5a8a7e333"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "34c8fe81d3d490a48ab6594dd94e316f"
server_key_iv: "5f305311204d5bd822453494c37a1d9e"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.370896 21406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:42.372177 21395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } attempt: 2
I20260809 19:32:42.372538 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.372420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49646)
0809 19:32:42.372477 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.372513 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:32:42.372951 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.372860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49652)
0809 19:32:42.372912 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.372935 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 19:32:42.373055 21395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:36477: Network error: Client connection negotiation failed: client connection to 127.31.57.253:36477: connect: Connection refused (error 111)
I20260809 19:32:42.403471 21406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:42.405408 21406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:42.405558 21406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:42.409721 21406 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.253:36477
I20260809 19:32:42.409775 21518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.253:36477 every 8 connection(s)
I20260809 19:32:42.410095 21406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
I20260809 19:32:42.410925 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:42.412328 21519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:42.419204 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 21406
I20260809 19:32:42.419293 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-1/wal/instance
I20260809 19:32:42.419567 31975 external_mini_cluster.cc:1428] Setting key 1ee2d4abf9feba8ea09c7367f3641b45
I20260809 19:32:42.420806 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.412958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.31.57.254:46767)
0809 19:32:42.413039 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.413041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.413197 (+ 156us) server_negotiation.cc:407] Connection header received
0809 19:32:42.413689 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.413691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.413706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.413784 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.414354 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.414859 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.415880 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.416077 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.416140 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.418068 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.418069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.418069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.418075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.419328 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.419911 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.419911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.419912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.419978 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.420250 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.420251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.420251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.420341 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.420388 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.420685 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.420744 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 19:32:42.421118 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.412908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46767 (local address 127.0.0.1:45952)
0809 19:32:42.413060 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.413092 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.413553 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.413864 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.413867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.414083 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.414265 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.414268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.414937 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.414938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.415793 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.415795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.416166 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.416167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.416264 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.416739 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.416746 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.417986 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.419464 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.419472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.419474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.419828 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.420046 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.420047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.420047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.420190 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.420402 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.420408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.420558 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.420666 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.420824 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 19:32:42.421697 21519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.57.252@KRBTEST.COM; defaulting to no policy
I20260809 19:32:42.429188 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.421945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49660)
0809 19:32:42.421996 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.422007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.422050 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.422349 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.422349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.422455 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.422500 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.422501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.423676 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.423677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.424075 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.424076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.424767 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.424767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.424827 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.425254 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.425258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.426337 (+ 1079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.428239 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.428240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.428240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.428392 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.428612 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.428612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.428612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.428660 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.428977 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.428978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.429024 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.429075 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.429137 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:32:42.429294 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.422031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.31.57.253:36477)
0809 19:32:42.422194 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.422199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.422224 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:32:42.422247 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.422249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.422263 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.422339 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.422576 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.423592 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.424174 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.424692 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.424769 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.426417 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.426424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.426425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.426432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.428158 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.428482 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.428485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.428485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.428545 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.428721 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.428722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.428722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.428906 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.428991 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.429117 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.429188 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260809 19:32:42.429925 21519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:32:42.430212 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.430094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48364)
0809 19:32:42.430142 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.430187 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:32:42.430372 21442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
I20260809 19:32:42.430747 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.430670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48378)
0809 19:32:42.430709 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.430731 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:32:42.430908 21519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:44393: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
I20260809 19:32:42.445623 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:44393
--webserver_interface=127.31.57.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:41513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:42.464193 21395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } attempt: 3
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:32:42.471315 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.464605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.31.57.253:36477)
0809 19:32:42.464674 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.464676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.464684 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:32:42.464752 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.464752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.464760 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.464813 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.465190 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.465666 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.466621 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.466811 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.466860 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.468822 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.468824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.468824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.468832 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.470210 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.470560 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.470561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.470562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.470617 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.470826 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.470826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.470827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.470930 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.471001 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.471160 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.471229 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:32:42.471331 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.464559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49674)
0809 19:32:42.464608 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.464621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.464683 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.464841 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.464842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.465023 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.465108 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.465110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.465743 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.465744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.466529 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.466532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.466886 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.466887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.466966 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.467431 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.467436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.468706 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.470306 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.470307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.470307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.470486 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.470691 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.470692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.470692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.470752 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.471006 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.471007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.471049 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.471136 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.471229 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:32:42.471905 21395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:32:42.472293 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.472187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48384)
0809 19:32:42.472233 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.472263 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:32:42.472669 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.472588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48400)
0809 19:32:42.472633 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.472655 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 19:32:42.472776 21395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:44393: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
I20260809 19:32:42.480091 21519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } attempt: 1
I20260809 19:32:42.480458 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.480337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48416)
0809 19:32:42.480391 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.480436 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 19:32:42.480808 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.480729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48420)
0809 19:32:42.480769 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.480792 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:32:42.480914 21519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:44393: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
I20260809 19:32:42.503890 21395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } attempt: 1
I20260809 19:32:42.504194 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.504090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48430)
0809 19:32:42.504140 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.504174 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:32:42.504516 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.504406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48444)
0809 19:32:42.504448 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.504494 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:32:42.504612 21395 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:44393: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
W20260809 19:32:42.528257 21528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:42.528478 21528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:42.528525 21528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:42.529729 21528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:42.529793 21528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:42.529822 21528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:42.529856 21528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:42.529903 21528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:42.531409 21528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:41513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46767,127.31.57.253:36477,127.31.57.252:44393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:44393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.57.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:42.531698 21528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:42.531975 21528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:42.536980 21542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:42.536988 21538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:42.536996 21544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:42.537200 21528 server_base.cc:1034] running on GCE node
I20260809 19:32:42.537473 21528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:42.537743 21528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:42.538892 21528 hybrid_clock.cc:648] HybridClock initialized: now 1786303962538884 us; error 32 us; skew 500 ppm
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:42.541527 21528 init.cc:377] Logged in from keytab as kudu/127.31.57.252@KRBTEST.COM (short username kudu)
I20260809 19:32:42.541936 21528 webserver.cc:460] Webserver started at http://127.31.57.252:44395/ using document root <none> and password file <none>
I20260809 19:32:42.542100 21528 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:42.542137 21528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:42.542219 21528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:42.543185 21528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data/instance:
uuid: "70bd28c341f541f88fdb953fcc376d99"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "e370a44adb464c55f0831e9cba84e904"
server_key_iv: "c4718ae89d3307268f99d65f67214222"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.543545 21528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal/instance:
uuid: "70bd28c341f541f88fdb953fcc376d99"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "e370a44adb464c55f0831e9cba84e904"
server_key_iv: "c4718ae89d3307268f99d65f67214222"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.544819 21528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:42.545485 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:42.545673 21528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:32:42.545780 21528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "70bd28c341f541f88fdb953fcc376d99"
format_stamp: "Formatted at 2026-08-09 19:32:42 on dist-test-slave-x4qh"
server_key: "e370a44adb464c55f0831e9cba84e904"
server_key_iv: "c4718ae89d3307268f99d65f67214222"
server_key_version: "encryptionkey@0"
I20260809 19:32:42.545862 21528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:42.549063 21519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } attempt: 2
I20260809 19:32:42.549492 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.549372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48454)
0809 19:32:42.549431 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.549464 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:32:42.549817 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.549737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48456)
0809 19:32:42.549778 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.549802 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:32:42.549932 21519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:44393: Network error: Client connection negotiation failed: client connection to 127.31.57.252:44393: connect: Connection refused (error 111)
I20260809 19:32:42.562130 21528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:42.564303 21528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:42.564499 21528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:42.568534 21528 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.252:44393
I20260809 19:32:42.568588 21637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.252:44393 every 8 connection(s)
I20260809 19:32:42.568948 21528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
I20260809 19:32:42.569815 21638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:42.570255 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 21528
I20260809 19:32:42.570376 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303811347120-31975-0/minicluster-data/master-2/wal/instance
I20260809 19:32:42.570556 31975 external_mini_cluster.cc:1428] Setting key c95a8e60f16c667fdaa934b690aec32e
I20260809 19:32:42.571784 21638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:32:42.579751 21395 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } attempt: 2
I20260809 19:32:42.580998 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.572428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.31.57.254:46767)
0809 19:32:42.572477 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.572480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.572617 (+ 137us) server_negotiation.cc:407] Connection header received
0809 19:32:42.572706 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.572707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.572719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.572787 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.573339 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.574042 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.575236 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.575570 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.575651 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.577931 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.577932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.577933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.577940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.579409 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.579877 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.579878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.579879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.579940 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.580229 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.580230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.580231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.580374 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.580460 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.580793 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.580885 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:32:42.581117 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.572332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:46767 (local address 127.0.0.1:45960)
0809 19:32:42.572512 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.572554 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.572613 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.572815 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.572817 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.573066 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.573232 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.573238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.574121 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.574122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.575157 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.575160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.575662 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.575664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.575744 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.576316 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.576327 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.577846 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.579506 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.579513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.579515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.579796 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.580043 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.580044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.580045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.580158 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.580462 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.580470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.580646 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.580761 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.580915 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260809 19:32:42.581730 21638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:32:42.581869 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.571043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.31.57.252:44393)
0809 19:32:42.571236 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.571244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.571275 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:32:42.571716 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.571723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.571773 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.571915 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.572585 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.573947 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.575041 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.575491 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.575609 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.578585 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.578592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.578592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.578597 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.580491 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.580868 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.580871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.580872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.580930 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.581158 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.581159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.581160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.581325 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.581437 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.581672 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.581742 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:42.587904 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.580002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48466)
0809 19:32:42.580049 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.580059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.580095 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.580299 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.580301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.580457 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.580497 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.580499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.581590 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.581591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.582254 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.582258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.582763 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.582764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.582852 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.583242 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.583248 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.584730 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.586785 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.586787 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.586787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.587018 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.587207 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.587208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.587208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.587276 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.587609 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.587610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.587652 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.587703 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.587796 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:32:42.588317 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.580039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.31.57.252:44393)
0809 19:32:42.580170 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.580171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.580177 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:32:42.580197 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.580197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.580208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.580277 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.580878 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.581521 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.582406 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.582681 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.582762 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.584814 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.584815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.584815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.584822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.586686 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.587085 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.587086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.587086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.587142 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.587347 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.587347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.587347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.587506 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.588148 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.588181 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.588235 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260809 19:32:42.589833 21395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: Bootstrap starting.
I20260809 19:32:42.590587 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.582398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.31.57.253:36477)
0809 19:32:42.582438 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.582440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.582447 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:42.582470 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.582471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.582482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.582530 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.582889 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.583579 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.584600 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.584856 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.584923 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.587736 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.587737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.587738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.587745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.589331 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.589720 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.589722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.589722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.589810 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.590030 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.590031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.590031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.590166 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.590248 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.590424 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.590510 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:32:42.590696 21395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:42.590901 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.581958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:36477 (local address 127.0.0.1:49688)
0809 19:32:42.581983 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.581993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.582029 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.582549 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.582549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.582702 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.582762 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.582763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.583677 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.583678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.584493 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.584497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.584976 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.584978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.585051 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.585952 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.585956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.587638 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.589419 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.589420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.589420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.589619 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.589890 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.589890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.589890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.589948 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.590241 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.590242 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.590283 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.590715 (+ 432us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.590825 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
I20260809 19:32:42.591073 21395 log.cc:824] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:42.591323 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.582648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.31.57.254:46767)
0809 19:32:42.582679 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.582681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.582687 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:32:42.582704 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.582705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.582711 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.582763 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.583447 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.584032 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.584871 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.585098 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.585160 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.588189 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.588190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.588191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.588197 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.589930 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.590359 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.590361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.590361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.590420 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.590657 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.590659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.590659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.590784 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.590936 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.591158 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.591231 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:32:42.591598 21638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:32:42.592221 21395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: No bootstrap required, opened a new log
I20260809 19:32:42.593906 21395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.594043 21395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:42.594101 21395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a737a82ddf4311851f916b6f517949, State: Initialized, Role: FOLLOWER
I20260809 19:32:42.594228 21395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.594548 21395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:42.594517 21648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.594635 21648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
W20260809 19:32:42.596376 21667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:32:42.596499 21667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:32:42.600004 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.591927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.31.57.253:36477)
0809 19:32:42.591974 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.591976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.591984 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:32:42.592036 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.592036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.592045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.592105 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.592548 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.593149 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.593703 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.593959 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.594032 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.596518 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.596519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.596519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.596525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.597960 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.598359 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.598360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.598361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.598410 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.598594 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.598596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.598596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.598738 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.599383 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.599799 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.599889 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:32:42.597627 21284 master_runner.cc:186] Error getting master registration for 127.31.57.253:36477: OK, instance_id { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" instance_seqno: 1786303962408447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:32:42.600868 21284 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:32:42.601769 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:32:42.602536 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.591826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48478)
0809 19:32:42.595479 (+ 3653us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.595488 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.595545 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.595720 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.595721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.595860 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.595886 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.595887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.596680 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.596681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.597366 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.597369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.597839 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.597840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.597888 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.598217 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.598222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.599726 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.601472 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.601472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.601472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.601658 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.601837 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.601838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.601839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.601909 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.602194 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.602195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.602238 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.602312 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.602409 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3645}
I20260809 19:32:42.602615 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.591877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.31.57.252:44393)
0809 19:32:42.595457 (+ 3580us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.595459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.595550 (+ 91us) server_negotiation.cc:407] Connection header received
0809 19:32:42.595613 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.595614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.595624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.595687 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.595962 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.596606 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.597447 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.597727 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.597797 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.599816 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.599817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.599817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.599823 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.601376 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.601722 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.601722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.601723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.601773 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.601977 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.601978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.601978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.602117 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.602179 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.602352 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.602479 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":3560}
I20260809 19:32:42.604272 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: Bootstrap starting.
I20260809 19:32:42.604806 21638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:42.605008 21638 log.cc:824] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:42.605943 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: No bootstrap required, opened a new log
I20260809 19:32:42.607153 21638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.607280 21638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:42.607324 21638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70bd28c341f541f88fdb953fcc376d99, State: Initialized, Role: FOLLOWER
I20260809 19:32:42.607448 21638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.607688 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.607729 21638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:42.607777 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:32:42.609349 21703 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:32:42.609444 21703 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:32:42.610553 21528 master_runner.cc:186] Error getting master registration for 127.31.57.253:36477: OK, instance_id { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" instance_seqno: 1786303962408447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:32:42.612790 21528 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:32:42.619817 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.603316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.31.57.254:46767)
0809 19:32:42.603349 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.603350 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.610114 (+ 6764us) server_negotiation.cc:407] Connection header received
0809 19:32:42.610199 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.610200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.610209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.610234 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.610930 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.611611 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.613365 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.613642 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.613678 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.615705 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.615705 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.615706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.615711 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.617822 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.618193 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.618194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.618194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.618250 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.618479 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.618480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.618480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.618589 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.618656 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.619644 (+ 988us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.619715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:32:42.620050 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.605623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.31.57.253:36477)
0809 19:32:42.605689 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.605691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.612612 (+ 6921us) server_negotiation.cc:407] Connection header received
0809 19:32:42.612699 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.612700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.612710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.612741 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.613325 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.614226 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.615226 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.615569 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.615616 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.617278 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.617279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.617279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.617297 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.618846 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.619195 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.619196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.619196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.619244 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.619472 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.619472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.619473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.619605 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.619666 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.619870 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.619954 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:32:42.624683 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.609058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.31.57.252:44393)
0809 19:32:42.609088 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.609090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.614068 (+ 4978us) server_negotiation.cc:407] Connection header received
0809 19:32:42.614179 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.614180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.614190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.614222 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.615964 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.616592 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.620716 (+ 4124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.620962 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.620996 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.622211 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.622212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.622213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.622219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.623547 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.623864 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.623865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.623865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.623914 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.624108 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.624109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.624109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.624206 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.624311 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.624506 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.624576 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:32:42.641058 21519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } attempt: 3
Aug 09 19:32:42 dist-test-slave-x4qh krb5kdc[21260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303962, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:32:42.647545 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.641295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:44393 (local address 127.0.0.1:48502)
0809 19:32:42.641348 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:42.641359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:42.641401 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:42.641583 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:42.641584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:42.641737 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:42.641776 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.641778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.642603 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.642604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.643054 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:42.643056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.643395 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.643396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:42.643452 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.643838 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:42.643842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:42.644945 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:42.646505 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.646506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.646507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.646684 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.646905 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:42.646906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:42.646906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:42.646963 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:42.647213 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:42.647214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:42.647254 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 19:32:42.647336 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:32:42.647453 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:32:42.647554 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:42.641369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.31.57.252:44393)
0809 19:32:42.641400 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:42.641402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:42.641409 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:42.641459 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:42.641459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:42.641467 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:42.641552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:42.642079 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.642521 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.643131 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:42.643297 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:42.643380 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:42.645042 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:42.645045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:42.645046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:42.645056 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:42.646377 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.646761 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.646762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.646762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.646837 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:42.647035 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:42.647036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:42.647037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:42.647138 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:42.647207 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:42.647411 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 19:32:42.647479 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:32:42.649093 21519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: Bootstrap starting.
I20260809 19:32:42.649763 21519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:42.650045 21519 log.cc:824] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:42.651052 21519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: No bootstrap required, opened a new log
I20260809 19:32:42.652436 21519 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.652565 21519 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:42.652642 21519 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fd6b0af1df44249a19a4a5a8a7e333, State: Initialized, Role: FOLLOWER
I20260809 19:32:42.652769 21519 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.653043 21707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.653105 21519 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:42.653170 21707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:32:42.654901 21731 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:32:42.654987 21731 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 19:32:42.656137 21445 tablet.cc:2369] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:32:42.656708 21406 master_runner.cc:263] No leader master found from master e0fd6b0af1df44249a19a4a5a8a7e333
I20260809 19:32:42.656752 21406 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:32:42.671883 21688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:42.671967 21688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.672151 21688 leader_election.cc:288] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a737a82ddf4311851f916b6f517949 (127.31.57.254:46767), e0fd6b0af1df44249a19a4a5a8a7e333 (127.31.57.253:36477)
I20260809 19:32:42.672580 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70bd28c341f541f88fdb953fcc376d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" is_pre_election: true
I20260809 19:32:42.672709 21474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70bd28c341f541f88fdb953fcc376d99 in term 0.
I20260809 19:32:42.673045 21353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70bd28c341f541f88fdb953fcc376d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a737a82ddf4311851f916b6f517949" is_pre_election: true
I20260809 19:32:42.673173 21353 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70bd28c341f541f88fdb953fcc376d99 in term 0.
I20260809 19:32:42.673813 21558 leader_election.cc:302] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70bd28c341f541f88fdb953fcc376d99, e0fd6b0af1df44249a19a4a5a8a7e333; no voters:
I20260809 19:32:42.674074 21688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:32:42.674124 21688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:32:42.674146 21688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:42.675025 21688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.675132 21688 leader_election.cc:288] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [CANDIDATE]: Term 1 election: Requested vote from peers 01a737a82ddf4311851f916b6f517949 (127.31.57.254:46767), e0fd6b0af1df44249a19a4a5a8a7e333 (127.31.57.253:36477)
I20260809 19:32:42.675451 21353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70bd28c341f541f88fdb953fcc376d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a737a82ddf4311851f916b6f517949"
I20260809 19:32:42.675518 21353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:42.676385 21353 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70bd28c341f541f88fdb953fcc376d99 in term 1.
I20260809 19:32:42.677224 21556 leader_election.cc:302] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a737a82ddf4311851f916b6f517949, 70bd28c341f541f88fdb953fcc376d99; no voters:
I20260809 19:32:42.677563 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70bd28c341f541f88fdb953fcc376d99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333"
I20260809 19:32:42.677629 21474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:42.678406 21474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70bd28c341f541f88fdb953fcc376d99 in term 1.
I20260809 19:32:42.678637 21688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:42.678786 21688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [term 1 LEADER]: Becoming Leader. State: Replica: 70bd28c341f541f88fdb953fcc376d99, State: Running, Role: LEADER
I20260809 19:32:42.678862 21688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } }
I20260809 19:32:42.679201 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70bd28c341f541f88fdb953fcc376d99. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.679260 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:42.679589 21739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:42.679778 21739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:42.681732 21474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [term 1 FOLLOWER]: Refusing update from remote peer 70bd28c341f541f88fdb953fcc376d99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:42.681950 21353 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [term 1 FOLLOWER]: Refusing update from remote peer 70bd28c341f541f88fdb953fcc376d99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:42.682365 21688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:42.682497 21688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:42.684296 21707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70bd28c341f541f88fdb953fcc376d99. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.684393 21707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:32:42.685169 21648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70bd28c341f541f88fdb953fcc376d99. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.685245 21648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:32:42.686640 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.686715 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:42.687090 21648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.687161 21648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:32:42.687417 21740 mvcc.cc:205] Tried to move back new op lower bound from 7316701031141933056 to 7316701031133220864. Current Snapshot: MvccSnapshot[applied={T|T < 7316701031141933056}]
I20260809 19:32:42.687690 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.687768 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:42.688201 21707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70bd28c341f541f88fdb953fcc376d99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a737a82ddf4311851f916b6f517949" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46767 } } peers { permanent_uuid: "e0fd6b0af1df44249a19a4a5a8a7e333" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 36477 } } peers { permanent_uuid: "70bd28c341f541f88fdb953fcc376d99" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 44393 } } }
I20260809 19:32:42.688268 21707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:32:42.688545 21741 mvcc.cc:205] Tried to move back new op lower bound from 7316701031141933056 to 7316701031133220864. Current Snapshot: MvccSnapshot[applied={T|T < 7316701031141933056}]
I20260809 19:32:42.688989 21745 mvcc.cc:205] Tried to move back new op lower bound from 7316701031141933056 to 7316701031133220864. Current Snapshot: MvccSnapshot[applied={T|T < 7316701031141933056}]
I20260809 19:32:42.690383 21739 catalog_manager.cc:1347] Generated new cluster ID: 7298e9338ddb4fbf8899eb032dada15b
I20260809 19:32:42.690424 21739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:32:42.706599 21739 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:32:42.707306 21739 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:42.712459 21739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70bd28c341f541f88fdb953fcc376d99: Generated new TSK 0
I20260809 19:32:42.712664 21739 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:32:43.596961 21667 catalog_manager.cc:1259] Loaded cluster ID: 7298e9338ddb4fbf8899eb032dada15b
I20260809 19:32:43.597040 21667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: loading cluster ID for follower catalog manager: success
I20260809 19:32:43.597770 21667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: acquiring CA information for follower catalog manager: success
I20260809 19:32:43.598022 21667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01a737a82ddf4311851f916b6f517949: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:32:43.655457 21731 catalog_manager.cc:1259] Loaded cluster ID: 7298e9338ddb4fbf8899eb032dada15b
I20260809 19:32:43.655531 21731 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: loading cluster ID for follower catalog manager: success
I20260809 19:32:43.656347 21731 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: acquiring CA information for follower catalog manager: success
I20260809 19:32:43.656658 21731 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0fd6b0af1df44249a19a4a5a8a7e333: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:32:43.760140 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 21284
I20260809 19:32:43.772862 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 21406
I20260809 19:32:43.784524 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 21528
2026-08-09T19:32:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2967 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:32:43 dist-test-slave-x4qh krb5kdc[21760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:32:43 dist-test-slave-x4qh krb5kdc[21760](info): set up 2 sockets
Aug 09 19:32:43 dist-test-slave-x4qh krb5kdc[21760](info): commencing operation
krb5kdc: starting...
W20260809 19:32:44.953931 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:32:45.065336 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:32797
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:32797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:45.139390 21784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:45.139555 21784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:45.139575 21784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:45.140542 21784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:45.140573 21784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:45.140587 21784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:45.140599 21784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:45.140609 21784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:45.142014 21784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:32797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:45.142187 21784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:45.142342 21784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:45.146940 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:45.146955 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:45.146967 21784 server_base.cc:1034] running on GCE node
W20260809 19:32:45.146966 21796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:45.147459 21784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:45.147673 21784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:45.148808 21784 hybrid_clock.cc:648] HybridClock initialized: now 1786303965148796 us; error 27 us; skew 500 ppm
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303965, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:45.151051 21784 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:45.151512 21784 webserver.cc:460] Webserver started at http://127.31.57.254:44949/ using document root <none> and password file <none>
I20260809 19:32:45.151700 21784 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:45.151736 21784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:45.151813 21784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:45.152654 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "0639fe3a4a154e1a9807eee453e8d5b6"
format_stamp: "Formatted at 2026-08-09 19:32:45 on dist-test-slave-x4qh"
server_key: "8fb431c40c0c042427d2927061b03c88"
server_key_iv: "8bbe8793ad4149d59d248a44487e2925"
server_key_version: "encryptionkey@0"
I20260809 19:32:45.152915 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "0639fe3a4a154e1a9807eee453e8d5b6"
format_stamp: "Formatted at 2026-08-09 19:32:45 on dist-test-slave-x4qh"
server_key: "8fb431c40c0c042427d2927061b03c88"
server_key_iv: "8bbe8793ad4149d59d248a44487e2925"
server_key_version: "encryptionkey@0"
I20260809 19:32:45.154044 21784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:32:45.154644 21807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:45.154834 21784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:32:45.154933 21784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "0639fe3a4a154e1a9807eee453e8d5b6"
format_stamp: "Formatted at 2026-08-09 19:32:45 on dist-test-slave-x4qh"
server_key: "8fb431c40c0c042427d2927061b03c88"
server_key_iv: "8bbe8793ad4149d59d248a44487e2925"
server_key_version: "encryptionkey@0"
I20260809 19:32:45.155011 21784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:45.165555 21784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:45.167075 21784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:45.167224 21784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:45.170920 21784 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:32797
I20260809 19:32:45.170969 21887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:32797 every 8 connection(s)
I20260809 19:32:45.171291 21784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:45.172068 21888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:45.173996 21888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6: Bootstrap starting.
I20260809 19:32:45.174526 21888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:45.174826 21888 log.cc:824] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:45.175554 21888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6: No bootstrap required, opened a new log
I20260809 19:32:45.177073 21888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } }
I20260809 19:32:45.177189 21888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:45.177210 21888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0639fe3a4a154e1a9807eee453e8d5b6, State: Initialized, Role: FOLLOWER
I20260809 19:32:45.177350 21888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } }
I20260809 19:32:45.177429 21888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:45.177451 21888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:45.177477 21888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:45.177930 21888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } }
I20260809 19:32:45.178040 21888 leader_election.cc:302] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0639fe3a4a154e1a9807eee453e8d5b6; no voters:
I20260809 19:32:45.178153 21888 leader_election.cc:288] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:32:45.178233 21893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:45.178381 21893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [term 1 LEADER]: Becoming Leader. State: Replica: 0639fe3a4a154e1a9807eee453e8d5b6, State: Running, Role: LEADER
I20260809 19:32:45.178421 21888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:45.178467 21893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } }
I20260809 19:32:45.179064 21896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0639fe3a4a154e1a9807eee453e8d5b6. Latest consensus state: current_term: 1 leader_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } } }
I20260809 19:32:45.179159 21896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:45.179255 21894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0639fe3a4a154e1a9807eee453e8d5b6" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 32797 } } }
I20260809 19:32:45.179396 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 21784
I20260809 19:32:45.179430 21894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:45.179495 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:32:45.179593 21903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:45.179694 31975 external_mini_cluster.cc:1428] Setting key a59e1bee26262e0e0df8b85a4b9a16a2
I20260809 19:32:45.180117 21903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:45.181931 21903 catalog_manager.cc:1347] Generated new cluster ID: 3e3a7923a41c4ac890ad85037cfd318e
I20260809 19:32:45.182024 21903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:45.191635 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:45.180315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.31.57.254:32797)
0809 19:32:45.180502 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:45.180532 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:45.180559 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:32:45.181004 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:45.181011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:45.181088 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:45.181233 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:45.182181 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:45.183558 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:45.184708 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:45.185313 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:45.185424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:45.188178 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:45.188184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:45.188184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:45.188199 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:45.190056 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:45.190440 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:45.190443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:45.190444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:45.190554 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:45.190743 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:45.190744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:45.190744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:45.191025 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:45.191180 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:45.191272 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 19:32:45.191433 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260809 19:32:45.193341 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:32:45.222144 21903 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:32:45.222792 21903 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:45.225203 21903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0639fe3a4a154e1a9807eee453e8d5b6: Generated new TSK 0
I20260809 19:32:45.225363 21903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:32:45.247185 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 21784
2026-08-09T19:32:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1473 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21929](info): set up 2 sockets
Aug 09 19:32:45 dist-test-slave-x4qh krb5kdc[21929](info): commencing operation
krb5kdc: starting...
W20260809 19:32:46.431653 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:32:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:32:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:32:46.540162 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:46.616290 21948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:46.616496 21948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:46.616530 21948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:46.617558 21948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:46.617621 21948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:46.617643 21948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:46.617663 21948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:46.617690 21948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:46.619150 21948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:46.619446 21948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:46.619668 21948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:46.624184 21955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.624192 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.624197 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:46.624562 21948 server_base.cc:1034] running on GCE node
I20260809 19:32:46.624783 21948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:46.625008 21948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:46.626144 21948 hybrid_clock.cc:648] HybridClock initialized: now 1786303966626132 us; error 30 us; skew 500 ppm
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:46.628477 21948 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:46.628921 21948 webserver.cc:460] Webserver started at http://127.31.57.254:34339/ using document root <none> and password file <none>
I20260809 19:32:46.629150 21948 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:46.629216 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:46.629367 21948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:46.630309 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "38558b6efd814a2795047aaa584c295b"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "23af247be7ff654cb6d73f3004876c6b"
server_key_iv: "d89cca1901917d0afc6c40a4bcbfc120"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.630653 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "38558b6efd814a2795047aaa584c295b"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "23af247be7ff654cb6d73f3004876c6b"
server_key_iv: "d89cca1901917d0afc6c40a4bcbfc120"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.631983 21948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:32:46.632656 21967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.632833 21948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:46.632930 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "38558b6efd814a2795047aaa584c295b"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "23af247be7ff654cb6d73f3004876c6b"
server_key_iv: "d89cca1901917d0afc6c40a4bcbfc120"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.633006 21948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:46.642307 21948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:46.644071 21948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:46.644230 21948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:46.648428 21948 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34331
I20260809 19:32:46.648483 22051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34331 every 8 connection(s)
I20260809 19:32:46.648873 21948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:46.649843 22052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:46.651994 22052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap starting.
I20260809 19:32:46.652529 22052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:46.652833 22052 log.cc:824] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:46.653517 22052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: No bootstrap required, opened a new log
I20260809 19:32:46.654970 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 21948
I20260809 19:32:46.655097 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:32:46.655293 31975 external_mini_cluster.cc:1428] Setting key 09850e51cdd54f669cfd151a2ead4641
I20260809 19:32:46.655310 22052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:46.655503 22052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:46.655570 22052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Initialized, Role: FOLLOWER
I20260809 19:32:46.655725 22052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:46.655834 22052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:46.655869 22052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:46.655915 22052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:46.657025 22052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:46.657155 22052 leader_election.cc:302] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38558b6efd814a2795047aaa584c295b; no voters:
I20260809 19:32:46.657341 22052 leader_election.cc:288] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:32:46.657392 22059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:46.657531 22059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 LEADER]: Becoming Leader. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Running, Role: LEADER
I20260809 19:32:46.657617 22059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:46.657622 22052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:46.657932 22061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:46.658028 22061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:46.657922 22059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38558b6efd814a2795047aaa584c295b. Latest consensus state: current_term: 1 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:46.658181 22059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:46.658835 22068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:46.659349 22068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:46.661767 22068 catalog_manager.cc:1347] Generated new cluster ID: 105699d5db7e432988ad56a298cae311
I20260809 19:32:46.661842 22068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:46.666460 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:46.655787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.31.57.254:34331)
0809 19:32:46.655963 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:46.655972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:46.656042 (+ 70us) server_negotiation.cc:407] Connection header received
0809 19:32:46.656166 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:46.656171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:46.656219 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:46.656320 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:46.657178 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.658504 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.659710 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.660218 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.660395 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:46.662884 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:46.662887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:46.662887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:46.662897 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:46.664725 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.665165 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.665167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.665167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.665275 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.665527 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.665527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.665528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.665861 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:46.666002 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:46.666144 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 19:32:46.666270 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260809 19:32:46.675328 22068 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:32:46.676023 22068 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:46.678853 22068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Generated new TSK 0
I20260809 19:32:46.678994 22068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:32:46.704257 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:46.781535 22087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:46.781733 22087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:46.781754 22087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:46.782943 22087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:46.782985 22087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:46.783034 22087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:32:46.784514 22087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:46.784754 22087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:46.784960 22087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:46.789894 22094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.789908 22095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.789865 22100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:46.790086 22087 server_base.cc:1034] running on GCE node
I20260809 19:32:46.790338 22087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:46.790575 22087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:46.791705 22087 hybrid_clock.cc:648] HybridClock initialized: now 1786303966791687 us; error 35 us; skew 500 ppm
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:46.794023 22087 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:32:46.794443 22087 webserver.cc:460] Webserver started at http://127.31.57.193:39355/ using document root <none> and password file <none>
I20260809 19:32:46.794677 22087 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:46.794739 22087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:46.794902 22087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:46.795914 22087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "2b498c1490d54e38be13d50d5eccffa6"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "33c441c8fc3660972b3609891e4bfa7b"
server_key_iv: "5d33d14abb3891b61181e034bc2d34ef"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.796293 22087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "2b498c1490d54e38be13d50d5eccffa6"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "33c441c8fc3660972b3609891e4bfa7b"
server_key_iv: "5d33d14abb3891b61181e034bc2d34ef"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.797571 22087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:32:46.798264 22116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.798483 22087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:46.798593 22087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "2b498c1490d54e38be13d50d5eccffa6"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "33c441c8fc3660972b3609891e4bfa7b"
server_key_iv: "5d33d14abb3891b61181e034bc2d34ef"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.798674 22087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:46.808262 22087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:46.809924 22087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:46.810092 22087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:46.810380 22087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:46.810676 22087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:46.810732 22087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.810774 22087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:46.810801 22087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.816560 22087 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:36763
I20260809 19:32:46.816608 22285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:36763 every 8 connection(s)
I20260809 19:32:46.816887 22087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:32:46.818890 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 22087
I20260809 19:32:46.819013 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:32:46.819203 31975 external_mini_cluster.cc:1428] Setting key 19ee6be2d61c4abd011c23a33461d051
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:46.826116 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:46.817618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:51771 (local address 127.31.57.254:34331)
0809 19:32:46.817678 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:46.817680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:46.817868 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:32:46.818282 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:46.818283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:46.818294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:46.818344 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:46.819452 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.820023 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.821060 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.821267 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.821352 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:46.823290 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:46.823291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:46.823291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:46.823298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:46.824723 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.825148 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.825149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.825149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.825207 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.825478 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.825479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.825480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.825598 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:46.825675 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:46.825941 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 19:32:46.826030 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:32:46.826320 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:46.817608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:51771)
0809 19:32:46.817756 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:46.817790 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:46.818190 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:46.818392 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:46.818395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:46.819076 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:46.819344 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:46.819349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.820114 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.820115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:46.820964 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:46.820967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.821366 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.821367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:46.821500 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:46.821976 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:46.821984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:46.823195 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:46.824827 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:46.824852 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:46.824853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:46.825070 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:46.825291 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:46.825292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:46.825293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:46.825405 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:46.825667 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:46.825675 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:46.825806 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 19:32:46.825905 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:32:46.826047 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260809 19:32:46.826772 22286 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:46.826877 22286 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:46.827065 22286 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
I20260809 19:32:46.827598 21998 ts_manager.cc:194] Registered new tserver with Master: 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193:36763)
I20260809 19:32:46.828241 21998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:51771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:32:46.843178 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:46.922287 22293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:46.922480 22293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:46.922505 22293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:46.923606 22293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:46.923652 22293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:46.923679 22293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:32:46.925410 22293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:46.925593 22293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:46.925800 22293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:46.930577 22300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.930629 22299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:46.930584 22303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:46.930842 22293 server_base.cc:1034] running on GCE node
I20260809 19:32:46.931061 22293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:46.931298 22293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:46.932502 22293 hybrid_clock.cc:648] HybridClock initialized: now 1786303966932495 us; error 55 us; skew 500 ppm
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:46.935107 22293 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:32:46.935561 22293 webserver.cc:460] Webserver started at http://127.31.57.194:46537/ using document root <none> and password file <none>
I20260809 19:32:46.935819 22293 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:46.935868 22293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:46.935954 22293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:46.936975 22293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "95b1a5524afa4167850e7240cec576b8"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "c50e0514b75df38677e79f0407694dbc"
server_key_iv: "daba80daa14344ea79289fc4a8898989"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.937321 22293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "95b1a5524afa4167850e7240cec576b8"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "c50e0514b75df38677e79f0407694dbc"
server_key_iv: "daba80daa14344ea79289fc4a8898989"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.938617 22293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:32:46.939299 22315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.939494 22293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:46.939576 22293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "95b1a5524afa4167850e7240cec576b8"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "c50e0514b75df38677e79f0407694dbc"
server_key_iv: "daba80daa14344ea79289fc4a8898989"
server_key_version: "encryptionkey@0"
I20260809 19:32:46.939627 22293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:46.949922 22293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:46.951646 22293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:46.951751 22293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:46.951912 22293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:46.952137 22293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:46.952167 22293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.952185 22293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:46.952198 22293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:46.957755 22293 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:45553
I20260809 19:32:46.957805 22496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:45553 every 8 connection(s)
I20260809 19:32:46.958052 22293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:32:46 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303966, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:46.965742 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:46.958651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:34483 (local address 127.31.57.254:34331)
0809 19:32:46.958710 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:46.958711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:46.958869 (+ 158us) server_negotiation.cc:407] Connection header received
0809 19:32:46.959277 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:46.959278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:46.959293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:46.959342 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:46.960190 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.960640 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.961483 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.961646 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.961692 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:46.963581 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:46.963582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:46.963583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:46.963589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:46.964664 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.965016 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.965017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.965017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.965065 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:46.965268 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:46.965268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:46.965269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:46.965365 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:46.965412 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:46.965623 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 19:32:46.965684 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:32:46.965914 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:46.958621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:34483)
0809 19:32:46.958766 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:46.958808 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:46.959202 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:46.959434 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:46.959437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:46.959953 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:46.960117 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:46.960121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.960714 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.960715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:46.961416 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:46.961418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:46.961714 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:46.961714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:46.961812 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:46.962262 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:46.962270 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:46.963505 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:46.964735 (+ 1230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:46.964741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:46.964742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:46.964951 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:46.965123 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:46.965124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:46.965124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:46.965215 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:46.965419 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:46.965423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:46.965523 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 19:32:46.965599 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:32:46.965704 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":101,"threads_started":1}
I20260809 19:32:46.966303 22497 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:46.966373 22497 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:46.966485 22497 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
I20260809 19:32:46.966758 21998 ts_manager.cc:194] Registered new tserver with Master: 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
I20260809 19:32:46.967087 21998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:34483
I20260809 19:32:46.967640 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 22293
I20260809 19:32:46.967741 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:32:46.967921 31975 external_mini_cluster.cc:1428] Setting key ef242f3e9d77d9ac5dcdb52e2d436796
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:32:46.990971 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:47.068140 22507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:47.068316 22507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:47.068336 22507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:47.069332 22507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:47.069370 22507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:47.069392 22507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:32:47.070683 22507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:47.070912 22507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:47.071125 22507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:47.075740 22516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.075742 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.075769 22518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.076164 22507 server_base.cc:1034] running on GCE node
I20260809 19:32:47.076362 22507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:47.076543 22507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:47.077677 22507 hybrid_clock.cc:648] HybridClock initialized: now 1786303967077664 us; error 27 us; skew 500 ppm
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:47.080029 22507 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:32:47.080437 22507 webserver.cc:460] Webserver started at http://127.31.57.195:34499/ using document root <none> and password file <none>
I20260809 19:32:47.080710 22507 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:47.080775 22507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:47.080910 22507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:32:47.081897 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "df1564d18c0b49c297dd54f167a5b417"
format_stamp: "Formatted at 2026-08-09 19:32:47 on dist-test-slave-x4qh"
server_key: "0e20f79904dd26908ea4a1774d4bc961"
server_key_iv: "a8b01835321f551cd39c65ebe7081fa6"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.082223 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "df1564d18c0b49c297dd54f167a5b417"
format_stamp: "Formatted at 2026-08-09 19:32:47 on dist-test-slave-x4qh"
server_key: "0e20f79904dd26908ea4a1774d4bc961"
server_key_iv: "a8b01835321f551cd39c65ebe7081fa6"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.083446 22507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:32:47.084091 22527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.084240 22507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:47.084329 22507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "df1564d18c0b49c297dd54f167a5b417"
format_stamp: "Formatted at 2026-08-09 19:32:47 on dist-test-slave-x4qh"
server_key: "0e20f79904dd26908ea4a1774d4bc961"
server_key_iv: "a8b01835321f551cd39c65ebe7081fa6"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.084419 22507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:47.094859 22507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:47.096545 22507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:47.096699 22507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:47.096925 22507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:47.097195 22507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:47.097246 22507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.097275 22507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:47.097307 22507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.102684 22507 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:35801
I20260809 19:32:47.102749 22693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:35801 every 8 connection(s)
I20260809 19:32:47.103061 22507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:32:47.105494 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 22507
I20260809 19:32:47.105581 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:32:47.105743 31975 external_mini_cluster.cc:1428] Setting key 240addb32ef70cbaa48e8b5d6761e34b
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:47.111725 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.103752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:50139 (local address 127.31.57.254:34331)
0809 19:32:47.103823 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.103825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.103968 (+ 143us) server_negotiation.cc:407] Connection header received
0809 19:32:47.104382 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.104383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.104392 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.104442 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.105242 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.106015 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.107148 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.107320 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.107442 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.109408 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.109409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.109410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.109415 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.110556 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.110932 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.110933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.110933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.110980 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.111182 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.111183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.111183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.111273 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.111329 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.111607 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.111668 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:32:47.111857 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.103702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:50139)
0809 19:32:47.103857 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.103887 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.104271 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.104487 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.104490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.104997 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:47.105169 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.105173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.106113 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.106114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.107078 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.107081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.107432 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.107433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.107540 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.108031 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:47.108043 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:47.109333 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:47.110626 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.110631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.110633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.110867 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.111038 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.111039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.111039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.111128 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.111336 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:47.111340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:47.111500 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.111578 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.111681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260809 19:32:47.112286 22694 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:47.112381 22694 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:47.112505 22694 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
I20260809 19:32:47.112869 21998 ts_manager.cc:194] Registered new tserver with Master: df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801)
I20260809 19:32:47.113174 21998 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:50139
I20260809 19:32:47.117496 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:32:47.125008 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.118769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.31.57.254:34331)
0809 19:32:47.118853 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.118855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.118927 (+ 72us) server_negotiation.cc:407] Connection header received
0809 19:32:47.118992 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.118993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.119009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.119058 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.119747 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.120202 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.120962 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.121146 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.121222 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.122404 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.122405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.122406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.122412 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.123846 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.124196 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.124197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.124198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.124253 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.124477 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.124478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.124478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.124622 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.124730 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.124836 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.124911 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 19:32:47.126646 21998 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:32:47.128330 21998 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:32:47.134325 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:35801 (local address 127.0.0.1:47652)
0809 19:32:47.130720 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.130731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.130770 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.131085 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.131088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.131090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.131164 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.131166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132152 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.132153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.133316 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.133319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.134025 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.134026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.134102 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.134113 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.134207 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.134259 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260809 19:32:47.134336 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.31.57.195:35801)
0809 19:32:47.130940 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.130944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.130951 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:47.130977 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.130978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.130989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.131073 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.131255 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132044 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.133407 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.133927 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.134019 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.134074 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.134224 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.134260 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260809 19:32:47.134670 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:45553 (local address 127.0.0.1:38612)
0809 19:32:47.130698 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.130709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.130768 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.131176 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.131177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.131179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.131239 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.131241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132551 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.132552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.133537 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.133540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.134451 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.134452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.134516 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.134519 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.134569 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.134605 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":84,"threads_started":1}
I20260809 19:32:47.134729 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.31.57.194:45553)
0809 19:32:47.130962 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.130965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.130973 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:32:47.131003 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.131004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.131015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.131110 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.131333 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132443 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.133621 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.134298 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.134358 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.134414 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.134602 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.134655 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":96,"threads_started":1}
I20260809 19:32:47.134838 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36763 (local address 127.0.0.1:50622)
0809 19:32:47.130583 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.130618 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.130679 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.131480 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.131480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.131483 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.131518 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.131520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132719 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.132720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.133547 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.133550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.134634 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.134635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.134686 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.134690 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.134721 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.134758 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
I20260809 19:32:47.135056 22599 tablet_service.cc:1467] Processing CreateTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a (DEFAULT_TABLE table=test-table [id=f4361960688a4105bc6057c2318f1eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.135464 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.130513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.31.57.193:36763)
0809 19:32:47.131283 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.131287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.131296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:32:47.131332 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.131334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.131348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.131479 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.131578 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.132576 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.133653 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.134289 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.135234 (+ 945us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.135304 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.135329 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.135388 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":89,"threads_started":1}
I20260809 19:32:47.135465 22599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f2e943a774fd3bacb6e2b0f561b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.135630 22402 tablet_service.cc:1467] Processing CreateTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a (DEFAULT_TABLE table=test-table [id=f4361960688a4105bc6057c2318f1eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.135975 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f2e943a774fd3bacb6e2b0f561b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.136174 22184 tablet_service.cc:1467] Processing CreateTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a (DEFAULT_TABLE table=test-table [id=f4361960688a4105bc6057c2318f1eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.136479 22184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36f2e943a774fd3bacb6e2b0f561b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.138389 22718 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Bootstrap starting.
I20260809 19:32:47.138952 22719 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Bootstrap starting.
I20260809 19:32:47.139096 22718 tablet_bootstrap.cc:654] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.139500 22720 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap starting.
I20260809 19:32:47.139619 22719 tablet_bootstrap.cc:654] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.139636 22718 log.cc:824] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.140079 22719 log.cc:824] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.140141 22720 tablet_bootstrap.cc:654] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.140414 22718 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: No bootstrap required, opened a new log
I20260809 19:32:47.140491 22718 ts_tablet_manager.cc:1397] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:32:47.140658 22720 log.cc:824] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.140796 22719 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: No bootstrap required, opened a new log
I20260809 19:32:47.140861 22719 ts_tablet_manager.cc:1397] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:32:47.142257 22718 raft_consensus.cc:348] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.142419 22718 raft_consensus.cc:374] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.142455 22718 raft_consensus.cc:729] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b1a5524afa4167850e7240cec576b8, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.142567 22718 consensus_queue.cc:260] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.142621 22719 raft_consensus.cc:348] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.142756 22719 raft_consensus.cc:374] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.142789 22719 raft_consensus.cc:729] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1564d18c0b49c297dd54f167a5b417, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.142859 22718 ts_tablet_manager.cc:1428] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:32:47.142897 22497 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
I20260809 19:32:47.142889 22719 consensus_queue.cc:260] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.143165 22694 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
I20260809 19:32:47.143209 22719 ts_tablet_manager.cc:1428] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:32:47.143460 22720 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: No bootstrap required, opened a new log
I20260809 19:32:47.143532 22720 ts_tablet_manager.cc:1397] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 19:32:47.145010 22720 raft_consensus.cc:348] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.145124 22720 raft_consensus.cc:374] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.145144 22720 raft_consensus.cc:729] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.145246 22720 consensus_queue.cc:260] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.145465 22720 ts_tablet_manager.cc:1428] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:32:47.145545 22286 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
W20260809 19:32:47.208537 22501 tablet.cc:2369] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:32:47.290107 22728 raft_consensus.cc:482] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:47.290243 22728 raft_consensus.cc:504] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.290632 22728 leader_election.cc:288] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553), df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801)
I20260809 19:32:47.293324 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.290776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:45553 (local address 127.31.57.193:42113)
0809 19:32:47.290833 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.290846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.290893 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.291055 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.291056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.291058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.291137 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.291138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.291785 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.291787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.292483 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.292485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.293055 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.293057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.293136 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.293138 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.293200 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.293252 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260809 19:32:47.293325 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.290848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:42113 (local address 127.31.57.194:45553)
0809 19:32:47.290895 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.290897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.290907 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:32:47.290942 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.290943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.290953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.291034 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.291211 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.291694 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.292591 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.292962 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.293043 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.293051 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.293220 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.293261 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:32:47.293504 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.290915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:43261 (local address 127.31.57.195:35801)
0809 19:32:47.290956 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.290958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.291060 (+ 102us) server_negotiation.cc:407] Connection header received
0809 19:32:47.291130 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.291131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.291147 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.291207 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.291394 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.291849 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.292726 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.293104 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.293189 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.293197 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.293372 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.293420 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:32:47.293674 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.290810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:35801 (local address 127.31.57.193:43261)
0809 19:32:47.290987 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.290995 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.291070 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.291213 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.291214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.291216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.291295 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.291297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.291947 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.291948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.292633 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.292635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.293202 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.293203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.293298 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.293301 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.293492 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.293536 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260809 19:32:47.293658 22428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f2e943a774fd3bacb6e2b0f561b8a" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b1a5524afa4167850e7240cec576b8" is_pre_election: true
I20260809 19:32:47.293833 22428 raft_consensus.cc:2454] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 0.
I20260809 19:32:47.294054 22627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f2e943a774fd3bacb6e2b0f561b8a" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1564d18c0b49c297dd54f167a5b417" is_pre_election: true
I20260809 19:32:47.294103 22122 leader_election.cc:302] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6, 95b1a5524afa4167850e7240cec576b8; no voters:
I20260809 19:32:47.294185 22627 raft_consensus.cc:2454] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 0.
I20260809 19:32:47.294229 22728 raft_consensus.cc:2781] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:32:47.294296 22728 raft_consensus.cc:482] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:32:47.294325 22728 raft_consensus.cc:3037] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:47.295024 22728 raft_consensus.cc:504] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.295171 22728 leader_election.cc:288] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 election: Requested vote from peers 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553), df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801)
I20260809 19:32:47.295316 22428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f2e943a774fd3bacb6e2b0f561b8a" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b1a5524afa4167850e7240cec576b8"
I20260809 19:32:47.295408 22627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36f2e943a774fd3bacb6e2b0f561b8a" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1564d18c0b49c297dd54f167a5b417"
I20260809 19:32:47.295446 22428 raft_consensus.cc:3037] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:47.295491 22627 raft_consensus.cc:3037] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:47.296159 22428 raft_consensus.cc:2454] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 1.
I20260809 19:32:47.296161 22627 raft_consensus.cc:2454] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 1.
I20260809 19:32:47.296325 22117 leader_election.cc:302] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6, df1564d18c0b49c297dd54f167a5b417; no voters:
I20260809 19:32:47.296422 22728 raft_consensus.cc:2781] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:47.296571 22728 raft_consensus.cc:686] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 LEADER]: Becoming Leader. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Running, Role: LEADER
I20260809 19:32:47.296696 22728 consensus_queue.cc:237] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.297529 21998 catalog_manager.cc:5495] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "2b498c1490d54e38be13d50d5eccffa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260809 19:32:47.316092 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.314001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.31.57.193:36763)
0809 19:32:47.314059 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.314065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.314072 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:47.314098 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.314098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.314112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.314170 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:32:47.314272 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.314673 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.315360 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.315580 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.315630 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.315776 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:32:47.315866 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:32:47.315924 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.315964 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.315991 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 19:32:47.317586 22288 tablet.cc:2369] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:32:47.317639 22627 raft_consensus.cc:1261] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Refusing update from remote peer 2b498c1490d54e38be13d50d5eccffa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:47.317690 22428 raft_consensus.cc:1261] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Refusing update from remote peer 2b498c1490d54e38be13d50d5eccffa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:47.318019 22728 consensus_queue.cc:1035] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:47.318202 22731 consensus_queue.cc:1035] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:47.320824 22738 mvcc.cc:205] Tried to move back new op lower bound from 7316701050130669568 to 7316701050047885312. Current Snapshot: MvccSnapshot[applied={T|T < 7316701050130669568}]
I20260809 19:32:47.323324 21998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:47.323443 21998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:47.324076 21998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Sending DeleteTablet for 3 replicas of tablet f36f2e943a774fd3bacb6e2b0f561b8a
I20260809 19:32:47.324409 22184 tablet_service.cc:1514] Processing DeleteTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:47 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50622
I20260809 19:32:47.324451 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 22087
I20260809 19:32:47.324468 22599 tablet_service.cc:1514] Processing DeleteTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:47 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:47652
I20260809 19:32:47.330377 22402 tablet_service.cc:1514] Processing DeleteTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:47 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:38612
I20260809 19:32:47.331411 22751 tablet_replica.cc:330] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: stopping tablet replica
I20260809 19:32:47.331480 22750 tablet_replica.cc:330] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: stopping tablet replica
I20260809 19:32:47.331514 22751 raft_consensus.cc:2229] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:47.331574 22750 raft_consensus.cc:2229] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:47.331645 22751 raft_consensus.cc:2258] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:47.331663 22750 raft_consensus.cc:2258] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:47.332062 22751 ts_tablet_manager.cc:1905] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:47.333396 22750 ts_tablet_manager.cc:1905] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:47.333613 22751 ts_tablet_manager.cc:1918] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:47.333698 22751 log.cc:1196] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/f36f2e943a774fd3bacb6e2b0f561b8a
I20260809 19:32:47.333941 22751 ts_tablet_manager.cc:1939] T f36f2e943a774fd3bacb6e2b0f561b8a P 95b1a5524afa4167850e7240cec576b8: Deleting consensus metadata
I20260809 19:32:47.334314 21975 catalog_manager.cc:4841] TS 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): tablet f36f2e943a774fd3bacb6e2b0f561b8a (table test-table [id=f4361960688a4105bc6057c2318f1eab]) successfully deleted
I20260809 19:32:47.335309 22750 ts_tablet_manager.cc:1918] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:47.335410 22750 log.cc:1196] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/f36f2e943a774fd3bacb6e2b0f561b8a
I20260809 19:32:47.335704 22750 ts_tablet_manager.cc:1939] T f36f2e943a774fd3bacb6e2b0f561b8a P df1564d18c0b49c297dd54f167a5b417: Deleting consensus metadata
I20260809 19:32:47.336143 21968 catalog_manager.cc:4841] TS df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): tablet f36f2e943a774fd3bacb6e2b0f561b8a (table test-table [id=f4361960688a4105bc6057c2318f1eab]) successfully deleted
W20260809 19:32:47.338130 21968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:36763 (error 108)
I20260809 19:32:47.338390 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 22293
I20260809 19:32:47.338569 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.338456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36763 (local address 127.0.0.1:50638)
0809 19:32:47.338508 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.338538 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:32:47.345160 21968 catalog_manager.cc:4584] TS 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193:36763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f36f2e943a774fd3bacb6e2b0f561b8a: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36763: connect: Connection refused (error 111)
I20260809 19:32:47.350312 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 22507
I20260809 19:32:47.361408 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 21948
I20260809 19:32:47.375623 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--webserver_interface=127.31.57.254
--webserver_port=34339
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:47.454881 22754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:47.455068 22754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:47.455091 22754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:47.456218 22754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:47.456257 22754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:47.456270 22754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:47.456282 22754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:47.456293 22754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:47.457556 22754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=34339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:47.457721 22754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:47.457877 22754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:47.467671 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.467671 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.467698 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.467825 22754 server_base.cc:1034] running on GCE node
I20260809 19:32:47.468079 22754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:47.468400 22754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:47.469556 22754 hybrid_clock.cc:648] HybridClock initialized: now 1786303967469546 us; error 27 us; skew 500 ppm
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:47.475925 22754 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:47.476383 22754 webserver.cc:460] Webserver started at http://127.31.57.254:34339/ using document root <none> and password file <none>
I20260809 19:32:47.476560 22754 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:47.476606 22754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:47.477828 22754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:47.478425 22774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.478586 22754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:47.478693 22754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "38558b6efd814a2795047aaa584c295b"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "23af247be7ff654cb6d73f3004876c6b"
server_key_iv: "d89cca1901917d0afc6c40a4bcbfc120"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.479082 22754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:47.496608 22754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:47.498266 22754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:47.498374 22754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:47.502110 22754 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34331
I20260809 19:32:47.502173 22849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34331 every 8 connection(s)
I20260809 19:32:47.502569 22754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:47.504189 22851 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:32:47.504858 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap starting.
I20260809 19:32:47.510854 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 22754
I20260809 19:32:47.511161 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:36763
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=39355
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:47.515827 22851 log.cc:824] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.517903 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:32:47.518204 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap complete.
I20260809 19:32:47.520591 22851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:47.520802 22851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.520961 22851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:47.521018 22851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:47.521076 22851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:47.521126 22851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:32:47.522138 22851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:47.522226 22851 leader_election.cc:302] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38558b6efd814a2795047aaa584c295b; no voters:
I20260809 19:32:47.522465 22851 leader_election.cc:288] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:32:47.522502 22856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:32:47.522631 22856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 LEADER]: Becoming Leader. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Running, Role: LEADER
I20260809 19:32:47.522739 22851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:47.522751 22856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:47.523123 22857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:47.523156 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38558b6efd814a2795047aaa584c295b. Latest consensus state: current_term: 2 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:47.523277 22857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:47.523296 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:47.523640 22868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:47.524536 22868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:47.524683 22868 catalog_manager.cc:1259] Loaded cluster ID: 105699d5db7e432988ad56a298cae311
I20260809 19:32:47.524755 22868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:32:47.525208 22868 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 19:32:47.530022 22868 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 19:32:47.532400 22868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:32:47.532639 22868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Loaded TSK: 0
I20260809 19:32:47.532871 22868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:32:47.593627 22854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:47.593856 22854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:47.593896 22854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:47.594998 22854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:47.595059 22854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:47.595103 22854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:32:47.596601 22854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:36763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=39355
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:47.596840 22854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:47.597046 22854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:47.601590 22890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.601605 22896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.601578 22891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.601790 22854 server_base.cc:1034] running on GCE node
I20260809 19:32:47.601985 22854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:47.602236 22854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:47.603433 22854 hybrid_clock.cc:648] HybridClock initialized: now 1786303967603410 us; error 36 us; skew 500 ppm
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:47.606401 22854 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:32:47.606930 22854 webserver.cc:460] Webserver started at http://127.31.57.193:39355/ using document root <none> and password file <none>
I20260809 19:32:47.607163 22854 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:47.607239 22854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:47.608932 22854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:47.609640 22905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.609843 22854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:47.609943 22854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "2b498c1490d54e38be13d50d5eccffa6"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "33c441c8fc3660972b3609891e4bfa7b"
server_key_iv: "5d33d14abb3891b61181e034bc2d34ef"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.610332 22854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:47.641558 22854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:47.643810 22854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:47.643953 22854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:47.644207 22854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:47.644646 22915 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:32:47.646034 22854 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:32:47.646090 22854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:32:47.646112 22854 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:32:47.646719 22854 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:32:47.646751 22854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:32:47.646842 22915 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap starting.
I20260809 19:32:47.653630 22854 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:36763
I20260809 19:32:47.653718 23063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:36763 every 8 connection(s)
I20260809 19:32:47.654017 22854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:32:47.656117 22915 log.cc:824] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.657403 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 22854
I20260809 19:32:47.657682 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:45553
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=46537
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:47.658391 22915 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:32:47.658660 22915 tablet_bootstrap.cc:492] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap complete.
I20260809 19:32:47.659899 22915 ts_tablet_manager.cc:1397] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
I20260809 19:32:47.661255 22915 raft_consensus.cc:348] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.661579 22915 raft_consensus.cc:729] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.661727 22915 consensus_queue.cc:260] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } }
I20260809 19:32:47.662019 22915 ts_tablet_manager.cc:1428] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:47.666157 23070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.654712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:51853)
0809 19:32:47.654951 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.654992 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.655507 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.655998 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.656001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.656292 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:47.656470 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.656474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.657903 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.657904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.659167 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.659171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.660174 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.660175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.660338 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.660895 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:47.660905 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:47.662610 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:47.664517 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.664524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.664526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.664742 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.665061 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.665062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.665062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.665173 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.665562 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:47.665568 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:47.665697 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.665803 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.665971 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
I20260809 19:32:47.666158 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.654828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:51853 (local address 127.31.57.254:34331)
0809 19:32:47.655529 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.655539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.655582 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:32:47.655724 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.655730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.655786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.656000 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.656593 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.657742 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.659262 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.659945 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.660062 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.662721 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.662725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.662725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.662736 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.664414 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.664848 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.664851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.664852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.664976 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.665261 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.665262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.665263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.665479 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.665568 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.665859 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.665982 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":121,"threads_started":1}
I20260809 19:32:47.666795 23064 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:47.666899 23064 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:47.667078 23064 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
I20260809 19:32:47.667821 22801 ts_manager.cc:194] Registered new tserver with Master: 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193:36763)
I20260809 19:32:47.668820 22801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:51853
I20260809 19:32:47.669813 23064 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:32:47.677023 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.668404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36763 (local address 127.0.0.1:50642)
0809 19:32:47.668548 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.668585 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.668644 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.669416 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.669423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.669721 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:47.669770 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.669772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.670951 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.670952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.671849 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.671857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.672482 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.672483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.672542 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.672934 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:47.672938 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:47.674249 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:47.676065 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.676071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.676073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.676254 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.676455 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.676456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.676456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.676505 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.676786 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:47.676792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:47.676829 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.676889 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.676967 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 19:32:47.677213 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.669102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.31.57.193:36763)
0809 19:32:47.669254 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.669258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.669269 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:32:47.669307 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.669309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.669322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.669390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.669855 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.670873 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.671971 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.672398 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.672461 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.674338 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.674345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.674346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.674352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.675982 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.676336 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.676339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.676339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.676395 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.676560 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.676561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.676561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.676728 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.676798 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.676938 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.677023 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260809 19:32:47.677561 22972 tablet_service.cc:1514] Processing DeleteTablet for tablet f36f2e943a774fd3bacb6e2b0f561b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:47 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50642
I20260809 19:32:47.678076 23080 tablet_replica.cc:330] stopping tablet replica
I20260809 19:32:47.678155 23080 raft_consensus.cc:2229] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:47.678234 23080 raft_consensus.cc:2258] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:47.678486 23080 ts_tablet_manager.cc:1905] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:47.680531 23080 ts_tablet_manager.cc:1918] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:47.680590 23080 log.cc:1196] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/f36f2e943a774fd3bacb6e2b0f561b8a
I20260809 19:32:47.680816 23080 ts_tablet_manager.cc:1939] T f36f2e943a774fd3bacb6e2b0f561b8a P 2b498c1490d54e38be13d50d5eccffa6: Deleting consensus metadata
I20260809 19:32:47.681299 22775 catalog_manager.cc:4841] TS 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193:36763): tablet f36f2e943a774fd3bacb6e2b0f561b8a (table test-table [id=f4361960688a4105bc6057c2318f1eab]) successfully deleted
W20260809 19:32:47.741418 23073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:47.741597 23073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:47.741619 23073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:47.742749 23073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:47.742796 23073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:47.742848 23073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:32:47.744491 23073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:45553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=46537
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:47.744699 23073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:47.744918 23073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:47.749719 23088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.749861 23073 server_base.cc:1034] running on GCE node
W20260809 19:32:47.749730 23085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.749730 23086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.750162 23073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:47.750424 23073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:47.751582 23073 hybrid_clock.cc:648] HybridClock initialized: now 1786303967751560 us; error 34 us; skew 500 ppm
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:47.753978 23073 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:32:47.754395 23073 webserver.cc:460] Webserver started at http://127.31.57.194:46537/ using document root <none> and password file <none>
I20260809 19:32:47.754565 23073 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:47.754613 23073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:47.755896 23073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:47.756397 23100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.756597 23073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:47.756680 23073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "95b1a5524afa4167850e7240cec576b8"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "c50e0514b75df38677e79f0407694dbc"
server_key_iv: "daba80daa14344ea79289fc4a8898989"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.756954 23073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:47.773833 23073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:47.775938 23073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:47.776068 23073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:47.776232 23073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:47.776461 23073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:47.776489 23073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.776508 23073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:47.776520 23073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.782003 23073 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:45553
I20260809 19:32:47.782065 23268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:45553 every 8 connection(s)
I20260809 19:32:47.782402 23073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:32:47.783861 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 23073
I20260809 19:32:47.784185 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:35801
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=34499
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:47.792734 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.783204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:43139)
0809 19:32:47.783357 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.783409 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.784047 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.784321 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.784326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.785130 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:47.785393 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.785399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.786120 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.786121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.786903 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.786906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.787481 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.787482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.787588 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.788062 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:47.788069 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:47.789492 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:47.791481 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.791487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.791489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.791686 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.791907 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.791908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.791908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.791998 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.792302 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:47.792307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:47.792410 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.792483 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.792592 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260809 19:32:47.792768 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.783181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:43139 (local address 127.31.57.254:34331)
0809 19:32:47.783233 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.783235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.783482 (+ 247us) server_negotiation.cc:407] Connection header received
0809 19:32:47.784164 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.784165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.784178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.784228 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.785483 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.785980 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.786998 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.787361 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.787467 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.789599 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.789601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.789602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.789610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.791389 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.791763 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.791765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.791765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.791830 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.792068 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.792068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.792069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.792223 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.792305 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.792534 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.792637 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:32:47.793329 23269 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:47.793409 23269 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:47.793601 23269 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
I20260809 19:32:47.794044 22801 ts_manager.cc:194] Registered new tserver with Master: 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
I20260809 19:32:47.794579 22801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:43139
W20260809 19:32:47.864415 23274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:47.864599 23274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:47.864621 23274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:47.865667 23274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:47.865702 23274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:47.865726 23274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:32:47.866983 23274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:35801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=34499
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:47.867138 23274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:47.867286 23274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:47.871922 23290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.871964 23287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:47.871933 23288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:47.871960 23274 server_base.cc:1034] running on GCE node
I20260809 19:32:47.872361 23274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:47.872594 23274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:47.873764 23274 hybrid_clock.cc:648] HybridClock initialized: now 1786303967873756 us; error 39 us; skew 500 ppm
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:47.876233 23274 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:32:47.876574 23274 webserver.cc:460] Webserver started at http://127.31.57.195:34499/ using document root <none> and password file <none>
I20260809 19:32:47.876734 23274 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:47.876780 23274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:47.877959 23274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:47.878505 23298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.878650 23274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:47.878727 23274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "df1564d18c0b49c297dd54f167a5b417"
format_stamp: "Formatted at 2026-08-09 19:32:47 on dist-test-slave-x4qh"
server_key: "0e20f79904dd26908ea4a1774d4bc961"
server_key_iv: "a8b01835321f551cd39c65ebe7081fa6"
server_key_version: "encryptionkey@0"
I20260809 19:32:47.878997 23274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:47.890157 23274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:47.891911 23274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:47.892009 23274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:47.892174 23274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:47.892386 23274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:47.892414 23274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.892432 23274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:47.892444 23274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.897886 23274 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:35801
I20260809 19:32:47.897957 23476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:35801 every 8 connection(s)
I20260809 19:32:47.898238 23274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:32:47.900355 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 23274
Aug 09 19:32:47 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303967, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:32:47.907070 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.900718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.31.57.254:34331)
0809 19:32:47.900834 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.900837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.900864 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:32:47.900956 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.900957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.900968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.901029 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.901541 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.901990 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.902855 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.903028 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.903103 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.904287 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.904288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.904288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.904296 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.905841 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.906193 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.906194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.906195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.906283 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.906482 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.906483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.906483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.906600 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.906764 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.906803 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.906881 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":48,"threads_started":1}
I20260809 19:32:47.907262 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.898860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:51299)
0809 19:32:47.898989 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.899029 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.899542 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.899765 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.899769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.900307 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:47.900543 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.900548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.901256 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.901257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.901981 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.901982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.902354 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.902355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.902475 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.902873 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:47.902880 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:47.904099 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:47.905859 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.905865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.905867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.906102 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.906322 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:47.906323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:47.906323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:47.906425 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:47.906687 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:47.906694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:47.906817 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.906912 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.907051 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260809 19:32:47.907335 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.898891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:51299 (local address 127.31.57.254:34331)
0809 19:32:47.898932 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.898934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.899101 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:32:47.899640 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.899642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.899654 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.899699 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:47.900660 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.901162 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.902068 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.902263 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.902333 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.904199 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:47.904201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:47.904201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:47.904207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:47.905756 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.906179 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.906180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.906181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.906245 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:47.906496 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:47.906496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:47.906497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:47.906616 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:47.906682 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.906944 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.907019 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:32:47.907797 23478 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34331
I20260809 19:32:47.907907 23478 heartbeater.cc:464] Registering TS with master...
I20260809 19:32:47.908077 23478 heartbeater.cc:511] Master 127.31.57.254:34331 requested a full tablet report, sending...
I20260809 19:32:47.908450 22800 ts_manager.cc:194] Registered new tserver with Master: df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801)
I20260809 19:32:47.908779 22800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:51299
I20260809 19:32:47.919075 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:32:47.921701 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.919401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.31.57.254:34331)
0809 19:32:47.919445 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.919447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.919562 (+ 115us) server_negotiation.cc:407] Connection header received
0809 19:32:47.919638 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.919639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.919672 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.919739 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:32:47.919854 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.920253 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.920926 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.921106 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.921176 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.921357 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:32:47.921481 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:32:47.921575 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.921612 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.921631 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:32:47.921991 22800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:32:47.923796 22800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:32:47.926069 22972 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 (DEFAULT_TABLE table=test-table [id=e7614213f86849df9a8743e69ecfc60e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.926270 22972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecaef65d1304c53bda4590fe6053c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.927912 22915 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap starting.
I20260809 19:32:47.928213 22915 tablet_bootstrap.cc:654] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.929762 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.925961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.31.57.194:45553)
0809 19:32:47.926258 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.926262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.926670 (+ 408us) server_negotiation.cc:407] Connection header received
0809 19:32:47.926728 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.926730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.926746 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.926832 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.926913 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.927595 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.928875 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.929395 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.929479 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.929532 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.929641 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.929685 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":86,"threads_started":1}
I20260809 19:32:47.929757 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.925706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:35801 (local address 127.0.0.1:47654)
0809 19:32:47.926045 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.926057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.926140 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.926483 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.926484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.926486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.926526 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.926528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.927647 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.927649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.928938 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.928940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.929520 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.929521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.929585 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.929589 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.929638 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.929685 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":4}
I20260809 19:32:47.929896 22915 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: No bootstrap required, opened a new log
I20260809 19:32:47.929948 22915 ts_tablet_manager.cc:1397] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:32:47.930053 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.925718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:45553 (local address 127.0.0.1:38622)
0809 19:32:47.926337 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:47.926349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:47.926412 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:47.926828 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:47.926829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:47.926830 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:47.926863 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.926864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.927739 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.927740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.928779 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:47.928782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.929478 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.929479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:47.929545 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.929567 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 19:32:47.929622 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:32:47.929678 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260809 19:32:47.930058 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:47.925787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.31.57.195:35801)
0809 19:32:47.926158 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:47.926162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:47.926172 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:32:47.926356 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:47.926358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:47.926386 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:47.926468 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:47.926612 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.927541 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.929014 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:47.929427 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:47.929516 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:47.929566 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:47.929919 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 19:32:47.929986 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":148,"threads_started":1}
I20260809 19:32:47.930083 22915 raft_consensus.cc:348] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.930151 22915 raft_consensus.cc:374] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.930200 22915 raft_consensus.cc:729] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.930256 22915 consensus_queue.cc:260] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.930348 22915 ts_tablet_manager.cc:1428] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:47.930773 23377 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 (DEFAULT_TABLE table=test-table [id=e7614213f86849df9a8743e69ecfc60e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.930804 23172 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 (DEFAULT_TABLE table=test-table [id=e7614213f86849df9a8743e69ecfc60e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:32:47.931109 23377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecaef65d1304c53bda4590fe6053c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.931123 23172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecaef65d1304c53bda4590fe6053c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:32:47.933614 23494 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Bootstrap starting.
I20260809 19:32:47.934203 23494 tablet_bootstrap.cc:654] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.934648 23494 log.cc:824] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.934928 23495 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Bootstrap starting.
I20260809 19:32:47.935364 23494 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: No bootstrap required, opened a new log
I20260809 19:32:47.935475 23494 ts_tablet_manager.cc:1397] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:32:47.935570 23495 tablet_bootstrap.cc:654] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Neither blocks nor log segments found. Creating new log.
I20260809 19:32:47.936036 23495 log.cc:824] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:47.936685 23495 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: No bootstrap required, opened a new log
I20260809 19:32:47.936759 23495 ts_tablet_manager.cc:1397] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:32:47.936885 23494 raft_consensus.cc:348] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.937017 23494 raft_consensus.cc:374] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.937038 23494 raft_consensus.cc:729] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1564d18c0b49c297dd54f167a5b417, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.937209 23494 consensus_queue.cc:260] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.937531 23494 ts_tablet_manager.cc:1428] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:32:47.937541 23478 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
I20260809 19:32:47.938632 23495 raft_consensus.cc:348] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.938757 23495 raft_consensus.cc:374] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:32:47.938805 23495 raft_consensus.cc:729] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b1a5524afa4167850e7240cec576b8, State: Initialized, Role: FOLLOWER
I20260809 19:32:47.938907 23495 consensus_queue.cc:260] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:47.939139 23495 ts_tablet_manager.cc:1428] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:32:47.939213 23269 heartbeater.cc:503] Master 127.31.57.254:34331 was elected leader, sending a full tablet report...
W20260809 19:32:48.033004 23271 tablet.cc:2369] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:32:48.148816 23479 tablet.cc:2369] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:32:48.154734 23068 tablet.cc:2369] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:32:48.229517 23075 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:48.229669 23075 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:48.230026 23075 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
I20260809 19:32:48.234009 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.230359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:48855 (local address 127.31.57.194:45553)
0809 19:32:48.230406 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:48.230409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:48.230423 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:32:48.230551 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:48.230552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:48.230562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:48.230632 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:48.230872 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.231608 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233111 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.233638 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233712 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:48.233720 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:48.233909 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:32:48.233944 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:32:48.234009 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.230359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:39843 (local address 127.31.57.195:35801)
0809 19:32:48.230406 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:48.230409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:48.230422 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:32:48.230551 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:48.230552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:48.230561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:48.230632 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:32:48.230856 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.231614 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233112 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.233638 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233712 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:48.233720 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:48.233909 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:32:48.233946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:32:48.234036 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.230230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:45553 (local address 127.31.57.193:48855)
0809 19:32:48.230368 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.230379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:48.230452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:48.230660 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:48.230661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:48.230665 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:48.230772 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:48.230784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.231696 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.231696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:48.233016 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:48.233021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.233728 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:48.233804 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:48.233828 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 19:32:48.233885 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:32:48.233943 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":116,"thread_start_us":62,"threads_started":1}
I20260809 19:32:48.234036 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.230230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:35801 (local address 127.31.57.193:39843)
0809 19:32:48.230368 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.230379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:48.230452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:48.230660 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:48.230661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:48.230664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:32:48.230752 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:48.230756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.231695 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.231696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:48.233016 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:48.233021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.233727 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.233728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:48.233801 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:48.233829 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 19:32:48.233885 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:32:48.233943 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":140,"thread_start_us":56,"threads_started":1}
I20260809 19:32:48.234421 23405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecaef65d1304c53bda4590fe6053c08" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1564d18c0b49c297dd54f167a5b417" is_pre_election: true
I20260809 19:32:48.234411 23199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecaef65d1304c53bda4590fe6053c08" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b1a5524afa4167850e7240cec576b8" is_pre_election: true
I20260809 19:32:48.234562 23199 raft_consensus.cc:2454] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 0.
I20260809 19:32:48.234562 23405 raft_consensus.cc:2454] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 0.
I20260809 19:32:48.234795 22910 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6, 95b1a5524afa4167850e7240cec576b8; no voters:
I20260809 19:32:48.234915 23075 raft_consensus.cc:2781] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:32:48.234989 23075 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:32:48.235011 23075 raft_consensus.cc:3037] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:48.235769 23075 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:48.235931 23075 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 election: Requested vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
I20260809 19:32:48.236101 23405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecaef65d1304c53bda4590fe6053c08" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1564d18c0b49c297dd54f167a5b417"
I20260809 19:32:48.236204 23405 raft_consensus.cc:3037] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:48.236891 23405 raft_consensus.cc:2454] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 1.
I20260809 19:32:48.237015 23199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecaef65d1304c53bda4590fe6053c08" candidate_uuid: "2b498c1490d54e38be13d50d5eccffa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b1a5524afa4167850e7240cec576b8"
I20260809 19:32:48.237080 22906 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6, df1564d18c0b49c297dd54f167a5b417; no voters:
I20260809 19:32:48.237093 23199 raft_consensus.cc:3037] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:32:48.237154 23075 raft_consensus.cc:2781] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:32:48.237243 23075 raft_consensus.cc:686] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 LEADER]: Becoming Leader. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Running, Role: LEADER
I20260809 19:32:48.237322 23075 consensus_queue.cc:237] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:48.238032 23199 raft_consensus.cc:2454] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b498c1490d54e38be13d50d5eccffa6 in term 1.
I20260809 19:32:48.238071 22800 catalog_manager.cc:5495] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "2b498c1490d54e38be13d50d5eccffa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } health_report { overall_health: HEALTHY } } }
I20260809 19:32:48.280516 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.278230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.31.57.193:36763)
0809 19:32:48.278438 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:48.278440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:48.278447 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:32:48.278479 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:48.278479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:48.278495 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:48.278562 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:32:48.278685 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.279109 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.279836 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:48.280010 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:48.280061 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:48.280195 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:32:48.280286 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:32:48.280339 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:48.280378 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:32:48.280395 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":144,"threads_started":1}
I20260809 19:32:48.282086 23199 raft_consensus.cc:1261] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Refusing update from remote peer 2b498c1490d54e38be13d50d5eccffa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:48.282110 23405 raft_consensus.cc:1261] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Refusing update from remote peer 2b498c1490d54e38be13d50d5eccffa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:32:48.282429 23075 consensus_queue.cc:1035] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:48.282588 23508 consensus_queue.cc:1035] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:32:48.285477 23517 mvcc.cc:205] Tried to move back new op lower bound from 7316701054080831488 to 7316701053900632064. Current Snapshot: MvccSnapshot[applied={T|T < 7316701054080831488}]
I20260809 19:32:48.285840 23518 mvcc.cc:205] Tried to move back new op lower bound from 7316701054080831488 to 7316701053900632064. Current Snapshot: MvccSnapshot[applied={T|T < 7316701054080831488}]
I20260809 19:32:48.287604 22801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:48.287675 22801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:32:48.288385 22801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Sending DeleteTablet for 3 replicas of tablet 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:48.288684 23172 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:48 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:38622
I20260809 19:32:48.288728 23377 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:48 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:47654
I20260809 19:32:48.288774 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 22854
I20260809 19:32:48.288865 22972 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecaef65d1304c53bda4590fe6053c08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:32:48 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50642
I20260809 19:32:48.292732 23534 tablet_replica.cc:330] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: stopping tablet replica
I20260809 19:32:48.292855 23534 raft_consensus.cc:2229] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:48.292951 23534 raft_consensus.cc:2258] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:48.295214 23534 ts_tablet_manager.cc:1905] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:48.296515 23535 tablet_replica.cc:330] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: stopping tablet replica
I20260809 19:32:48.296622 23535 raft_consensus.cc:2229] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:32:48.296729 23535 raft_consensus.cc:2258] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:32:48.297429 23534 ts_tablet_manager.cc:1918] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:48.297536 23534 log.cc:1196] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:48.297854 23534 ts_tablet_manager.cc:1939] T 5ecaef65d1304c53bda4590fe6053c08 P 95b1a5524afa4167850e7240cec576b8: Deleting consensus metadata
I20260809 19:32:48.298252 22778 catalog_manager.cc:4841] TS 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): tablet 5ecaef65d1304c53bda4590fe6053c08 (table test-table [id=e7614213f86849df9a8743e69ecfc60e]) successfully deleted
I20260809 19:32:48.298578 23535 ts_tablet_manager.cc:1905] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:32:48.299787 23535 ts_tablet_manager.cc:1918] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:32:48.299916 23535 log.cc:1196] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:48.300192 23535 ts_tablet_manager.cc:1939] T 5ecaef65d1304c53bda4590fe6053c08 P df1564d18c0b49c297dd54f167a5b417: Deleting consensus metadata
I20260809 19:32:48.300606 22775 catalog_manager.cc:4841] TS df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): tablet 5ecaef65d1304c53bda4590fe6053c08 (table test-table [id=e7614213f86849df9a8743e69ecfc60e]) successfully deleted
W20260809 19:32:48.302862 22775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:36763 (error 108)
I20260809 19:32:48.303150 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 23073
I20260809 19:32:48.303319 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.303182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36763 (local address 127.0.0.1:50664)
0809 19:32:48.303240 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.303284 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 19:32:48.303424 22775 catalog_manager.cc:4584] TS 2b498c1490d54e38be13d50d5eccffa6 (127.31.57.193:36763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ecaef65d1304c53bda4590fe6053c08: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36763: connect: Connection refused (error 111)
I20260809 19:32:48.313020 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 23274
I20260809 19:32:48.320276 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 22754
I20260809 19:32:48.330828 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--webserver_interface=127.31.57.254
--webserver_port=34339
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:48.405576 23537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:48.405759 23537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:48.405788 23537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:48.406862 23537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:32:48.406899 23537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:48.406912 23537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:48.406963 23537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:32:48.406984 23537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:32:48.408445 23537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=34339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:48.408669 23537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:48.408897 23537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:48.413347 23544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.413466 23537 server_base.cc:1034] running on GCE node
W20260809 19:32:48.413380 23548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.413380 23543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.413795 23537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:48.414121 23537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:48.415306 23537 hybrid_clock.cc:648] HybridClock initialized: now 1786303968415291 us; error 42 us; skew 500 ppm
Aug 09 19:32:48 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:48.417624 23537 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:32:48.418082 23537 webserver.cc:460] Webserver started at http://127.31.57.254:34339/ using document root <none> and password file <none>
I20260809 19:32:48.418298 23537 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:48.418362 23537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:48.419727 23537 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:48.420346 23559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.420526 23537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:32:48.420615 23537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "38558b6efd814a2795047aaa584c295b"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "23af247be7ff654cb6d73f3004876c6b"
server_key_iv: "d89cca1901917d0afc6c40a4bcbfc120"
server_key_version: "encryptionkey@0"
I20260809 19:32:48.420959 23537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:48.436551 23537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:48.438076 23537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:48.438223 23537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:48.442101 23537 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34331
I20260809 19:32:48.442132 23644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34331 every 8 connection(s)
I20260809 19:32:48.442461 23537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:32:48.443987 23646 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:32:48.444545 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap starting.
I20260809 19:32:48.445693 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 23537
I20260809 19:32:48.445945 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:36763
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=39355
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:48.453572 23646 log.cc:824] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:48.455796 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:32:48.456085 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b: Bootstrap complete.
I20260809 19:32:48.458271 23646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:48.458542 23646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Initialized, Role: FOLLOWER
I20260809 19:32:48.458705 23646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:48.458762 23646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:32:48.458832 23646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:32:48.458899 23646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:32:48.459920 23646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:48.460004 23646 leader_election.cc:302] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38558b6efd814a2795047aaa584c295b; no voters:
I20260809 19:32:48.460198 23646 leader_election.cc:288] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:32:48.460232 23652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:32:48.460353 23652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [term 3 LEADER]: Becoming Leader. State: Replica: 38558b6efd814a2795047aaa584c295b, State: Running, Role: LEADER
I20260809 19:32:48.460469 23646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:32:48.460454 23652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } }
I20260809 19:32:48.460747 23652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38558b6efd814a2795047aaa584c295b. Latest consensus state: current_term: 3 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:48.460824 23652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:48.460752 23654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "38558b6efd814a2795047aaa584c295b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38558b6efd814a2795047aaa584c295b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34331 } } }
I20260809 19:32:48.461030 23662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:32:48.461071 23654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38558b6efd814a2795047aaa584c295b [sys.catalog]: This master's current role is: LEADER
I20260809 19:32:48.461836 23662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:32:48.461963 23662 catalog_manager.cc:1259] Loaded cluster ID: 105699d5db7e432988ad56a298cae311
I20260809 19:32:48.461998 23662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 19:32:48.462378 23662 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7b1b3409ad google::LogMessage::Fail() at ??:0
@ 0x7f7b1b34388a google::LogMessage::SendToLog() at ??:0
@ 0x7f7b1b3404de google::LogMessage::Flush() at ??:0
@ 0x7f7b1b341a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7b232610f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7b2326a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7b1d20e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7b1d203954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7b1e0676db start_thread at ??:0
@ 0x7f7b19e3e61f clone at ??:0
W20260809 19:32:48.557538 23650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:48.557767 23650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:48.557808 23650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:48.558900 23650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:48.558970 23650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:48.559019 23650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:32:48.560539 23650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:36763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=39355
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:48.560822 23650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:48.561044 23650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:48.565974 23704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.566013 23702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.565984 23701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.566548 23650 server_base.cc:1034] running on GCE node
I20260809 19:32:48.566758 23650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:48.567001 23650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:48.568161 23650 hybrid_clock.cc:648] HybridClock initialized: now 1786303968568146 us; error 31 us; skew 500 ppm
Aug 09 19:32:48 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:48.570744 23650 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:32:48.571205 23650 webserver.cc:460] Webserver started at http://127.31.57.193:39355/ using document root <none> and password file <none>
I20260809 19:32:48.571491 23650 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:48.571563 23650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:48.572885 23650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:48.573632 23713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.573825 23650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:32:48.573927 23650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "2b498c1490d54e38be13d50d5eccffa6"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "33c441c8fc3660972b3609891e4bfa7b"
server_key_iv: "5d33d14abb3891b61181e034bc2d34ef"
server_key_version: "encryptionkey@0"
I20260809 19:32:48.574319 23650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:48.593700 23650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:48.596094 23650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:48.596246 23650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:48.596493 23650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:48.597002 23723 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:32:48.598421 23650 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:32:48.598483 23650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:32:48.598516 23650 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:32:48.599362 23650 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:32:48.599443 23650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:48.599539 23723 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap starting.
I20260809 19:32:48.606912 23650 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:36763
I20260809 19:32:48.606988 23878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:36763 every 8 connection(s)
I20260809 19:32:48.607311 23650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:32:48.609477 23723 log.cc:824] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:32:48.611325 23723 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:32:48.611603 23723 tablet_bootstrap.cc:492] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Bootstrap complete.
I20260809 19:32:48.612053 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 23650
I20260809 19:32:48.612357 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:45553
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=46537
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:32:48.612769 23723 ts_tablet_manager.cc:1397] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20260809 19:32:48.614319 23723 raft_consensus.cc:348] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:48.614550 23723 raft_consensus.cc:729] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b498c1490d54e38be13d50d5eccffa6, State: Initialized, Role: FOLLOWER
I20260809 19:32:48.614701 23723 consensus_queue.cc:260] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:48.615015 23723 ts_tablet_manager.cc:1428] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260809 19:32:48.666972 23885 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:32:48.607949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:59787)
0809 19:32:48.608161 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.608228 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:48.608914 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:48.666789 (+ 57875us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
W20260809 19:32:48.667227 23879 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:34331 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:34331: Client connection negotiation failed: client connection to 127.31.57.254:34331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:32:48.667601 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.667468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:35785)
0809 19:32:48.667519 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.667570 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:32:48.667908 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.667824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:55291)
0809 19:32:48.667865 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.667891 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:32:48.668042 23879 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:32:48.692591 23887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:48.692755 23887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:48.692807 23887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:48.693848 23887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:48.693884 23887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:48.693928 23887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:32:48.695302 23887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:45553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=46537
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:48.695549 23887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:48.695763 23887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:48.700346 23897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.700388 23887 server_base.cc:1034] running on GCE node
W20260809 19:32:48.700316 23900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.700330 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.700724 23887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:48.700949 23887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:48.702096 23887 hybrid_clock.cc:648] HybridClock initialized: now 1786303968702082 us; error 45 us; skew 500 ppm
Aug 09 19:32:48 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:48.704331 23887 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:32:48.704810 23887 webserver.cc:460] Webserver started at http://127.31.57.194:46537/ using document root <none> and password file <none>
I20260809 19:32:48.705046 23887 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:48.705116 23887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:48.706369 23887 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:48.706923 23910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.707082 23887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:48.707171 23887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "95b1a5524afa4167850e7240cec576b8"
format_stamp: "Formatted at 2026-08-09 19:32:46 on dist-test-slave-x4qh"
server_key: "c50e0514b75df38677e79f0407694dbc"
server_key_iv: "daba80daa14344ea79289fc4a8898989"
server_key_version: "encryptionkey@0"
I20260809 19:32:48.707504 23887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:48.725857 23887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:48.727620 23887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:48.727766 23887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:48.727977 23887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:48.728295 23887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:48.728353 23887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.728389 23887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:48.728413 23887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.734257 23887 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:45553
I20260809 19:32:48.734316 24086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:45553 every 8 connection(s)
I20260809 19:32:48.734614 23887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:32:48.735726 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.735312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:56235)
0809 19:32:48.735536 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.735604 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":132,"threads_started":1}
W20260809 19:32:48.736083 24087 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:34331 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:34331: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
I20260809 19:32:48.736322 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.736255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:48245)
0809 19:32:48.736279 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.736308 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260809 19:32:48.736573 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.736512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:52005)
0809 19:32:48.736531 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.736546 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 19:32:48.736688 24087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:32:48.738474 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 23887
I20260809 19:32:48.738734 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:35801
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:34331
--webserver_port=34499
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:32:48.814150 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:32:48.814339 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:32:48.814363 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:32:48.815478 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:32:48.815516 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:32:48.815541 24090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:32:48.816926 24090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:35801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=34499
--tserver_master_addrs=127.31.57.254:34331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:32:48.817152 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:32:48.817401 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:32:48.822057 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.822089 24096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:32:48.822119 24099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:32:48.822425 24090 server_base.cc:1034] running on GCE node
I20260809 19:32:48.822631 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:32:48.822897 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:32:48.824038 24090 hybrid_clock.cc:648] HybridClock initialized: now 1786303968824018 us; error 38 us; skew 500 ppm
Aug 09 19:32:48 dist-test-slave-x4qh krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:32:48.826442 24090 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:32:48.826817 24090 webserver.cc:460] Webserver started at http://127.31.57.195:34499/ using document root <none> and password file <none>
I20260809 19:32:48.827018 24090 fs_manager.cc:359] Metadata directory not provided
I20260809 19:32:48.827059 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:32:48.828230 24090 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:32:48.828775 24108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.828955 24090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:32:48.829041 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "df1564d18c0b49c297dd54f167a5b417"
format_stamp: "Formatted at 2026-08-09 19:32:47 on dist-test-slave-x4qh"
server_key: "0e20f79904dd26908ea4a1774d4bc961"
server_key_iv: "a8b01835321f551cd39c65ebe7081fa6"
server_key_version: "encryptionkey@0"
I20260809 19:32:48.829315 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:32:48.836172 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:32:48.837704 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:32:48.837852 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:32:48.838073 24090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:32:48.838359 24090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:32:48.838398 24090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.838418 24090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:32:48.838464 24090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:32:48.844167 24090 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:35801
I20260809 19:32:48.844201 24282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:35801 every 8 connection(s)
I20260809 19:32:48.844540 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:32:48.845434 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.845093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:43603)
0809 19:32:48.845244 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.845315 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260809 19:32:48.845781 24283 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:34331 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:34331: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
I20260809 19:32:48.846021 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.845938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:49343)
0809 19:32:48.845982 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.846007 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:32:48.846259 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:48.846215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:36737)
0809 19:32:48.846235 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:48.846250 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 19:32:48.846364 24283 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:32:48.853868 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 24090
I20260809 19:32:49.668581 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.668371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:48071)
0809 19:32:49.668491 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:49.668523 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 19:32:49.737342 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.737146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:45259)
0809 19:32:49.737259 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:49.737300 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260809 19:32:49.805688 24291 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:49.805809 24291 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:49.806106 24291 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
Aug 09 19:32:49 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.194@KRBTEST.COM
Aug 09 19:32:49 dist-test-slave-x4qh krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303968, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.195@KRBTEST.COM
I20260809 19:32:49.815841 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.806420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:36081 (local address 127.31.57.194:45553)
0809 19:32:49.806587 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:49.806596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:49.806624 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:32:49.807037 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:49.807039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:49.807083 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:49.807234 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:49.807990 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.808787 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.809822 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.810289 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.810391 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:49.812448 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:49.812452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:49.812452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:49.812463 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:49.814158 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814586 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:49.814590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:49.814590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:49.814689 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814941 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:49.814943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:49.814943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:49.815188 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:49.815291 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:49.815605 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 19:32:49.815718 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260809 19:32:49.815840 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.806357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:53825 (local address 127.31.57.195:35801)
0809 19:32:49.806519 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 19:32:49.806532 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 19:32:49.806563 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:32:49.807000 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:32:49.807003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:32:49.807052 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:32:49.807163 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:32:49.808005 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.808778 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.809791 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.810188 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.810267 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:49.812774 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:32:49.812777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:32:49.812777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:32:49.812787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:32:49.814244 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814623 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:49.814626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:49.814626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:49.814702 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814967 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:32:49.814968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:32:49.814968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:32:49.815188 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:32:49.815279 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:32:49.815610 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 19:32:49.815768 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":55,"threads_started":1}
I20260809 19:32:49.816026 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.806277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:45553 (local address 127.31.57.193:36081)
0809 19:32:49.806416 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:49.806425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:49.806470 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:49.807266 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:49.807268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:49.807673 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:49.807877 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:49.807882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.808917 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.808918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:49.809717 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:49.809720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.810374 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.810375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:49.810416 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:49.810966 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:49.810973 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:49.812343 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:49.814242 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:49.814251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:49.814511 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:49.814756 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:49.814757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:49.814876 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:49.815266 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:49.815273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:49.815432 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 19:32:49.815548 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 19:32:49.815945 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":321,"thread_start_us":57,"threads_started":1}
I20260809 19:32:49.816151 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.806231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:35801 (local address 127.31.57.193:53825)
0809 19:32:49.806273 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:49.806295 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 19:32:49.806357 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:32:49.807253 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:32:49.807256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:32:49.807672 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:32:49.807877 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:49.807882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.808889 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.808890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:49.809707 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:32:49.809710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:32:49.810273 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:32:49.810280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:32:49.810395 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:32:49.811130 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:32:49.811135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:32:49.812662 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:32:49.814339 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:49.814340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:49.814532 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:49.814804 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:32:49.814804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:32:49.814805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:32:49.814879 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:32:49.815276 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:32:49.815277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:32:49.815431 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 19:32:49.815548 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 19:32:49.815726 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":13}
W20260809 19:32:49.816738 23717 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
W20260809 19:32:49.816833 23714 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:49.816871 23714 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6; no voters: 95b1a5524afa4167850e7240cec576b8, df1564d18c0b49c297dd54f167a5b417
I20260809 19:32:49.816962 24291 raft_consensus.cc:2726] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:32:49.846794 24296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:49.846636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:56693)
0809 19:32:49.846729 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:49.846763 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":44,"threads_started":1}
I20260809 19:32:50.669304 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:50.668994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:43517)
0809 19:32:50.669220 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:50.669258 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":69,"threads_started":1}
I20260809 19:32:50.737884 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:50.737715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:36893)
0809 19:32:50.737824 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:50.737851 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260809 19:32:50.847349 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:50.847175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:37753)
0809 19:32:50.847290 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:50.847319 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260809 19:32:51.670099 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:51.669898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:49809)
0809 19:32:51.670020 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:51.670055 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260809 19:32:51.738487 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:51.738286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:48687)
0809 19:32:51.738422 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:51.738452 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260809 19:32:51.801579 24307 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:51.801669 24307 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:51.801883 24307 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
W20260809 19:32:51.802275 23717 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
W20260809 19:32:51.802358 23714 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:51.802443 23714 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6; no voters: 95b1a5524afa4167850e7240cec576b8, df1564d18c0b49c297dd54f167a5b417
I20260809 19:32:51.802532 24307 raft_consensus.cc:2726] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:32:51.847967 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:51.847798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:58397)
0809 19:32:51.847905 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:51.847936 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260809 19:32:52.670831 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:52.670590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:52231)
0809 19:32:52.670733 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:52.670785 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260809 19:32:52.739431 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:52.739116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:33417)
0809 19:32:52.739315 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:52.739346 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260809 19:32:52.848538 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:52.848351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:35973)
0809 19:32:52.848477 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:52.848509 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":53,"threads_started":1}
I20260809 19:32:53.671598 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:53.671282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:51527)
0809 19:32:53.671506 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:53.671554 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":150,"threads_started":1}
I20260809 19:32:53.740012 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:53.739817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:57809)
0809 19:32:53.739946 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:53.739977 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260809 19:32:53.849107 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:53.848908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:34277)
0809 19:32:53.849042 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:53.849072 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260809 19:32:53.866792 24319 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:53.866894 24319 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:53.867091 24319 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
W20260809 19:32:53.867483 23714 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
W20260809 19:32:53.867604 23717 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:53.867712 23717 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6; no voters: 95b1a5524afa4167850e7240cec576b8, df1564d18c0b49c297dd54f167a5b417
I20260809 19:32:53.867826 24319 raft_consensus.cc:2726] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:32:54.672230 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:54.672000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:59903)
0809 19:32:54.672129 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:54.672182 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260809 19:32:54.740612 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:54.740425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:56025)
0809 19:32:54.740547 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:54.740578 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260809 19:32:54.849645 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:54.849478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:53087)
0809 19:32:54.849583 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:54.849613 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260809 19:32:55.672885 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:55.672643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:54383)
0809 19:32:55.672790 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:55.672825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260809 19:32:55.741293 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:55.741105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:35597)
0809 19:32:55.741218 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:55.741251 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260809 19:32:55.850201 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:55.850025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:45245)
0809 19:32:55.850139 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:55.850168 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260809 19:32:56.673866 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:56.673629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:33647)
0809 19:32:56.673781 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:56.673819 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260809 19:32:56.742076 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:56.741888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:33959)
0809 19:32:56.742010 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:56.742042 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260809 19:32:56.850946 24335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:56.850754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:47377)
0809 19:32:56.850877 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:56.850912 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260809 19:32:57.575631 24336 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:32:57.575742 24336 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:32:57.575989 24336 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
W20260809 19:32:57.576367 23714 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
W20260809 19:32:57.576443 23717 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:32:57.576516 23717 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6; no voters: 95b1a5524afa4167850e7240cec576b8, df1564d18c0b49c297dd54f167a5b417
I20260809 19:32:57.576581 24336 raft_consensus.cc:2726] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:32:57.674526 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:57.674317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:52621)
0809 19:32:57.674447 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:57.674481 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260809 19:32:57.742822 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:57.742556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:43133)
0809 19:32:57.742743 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:57.742789 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":53,"threads_started":1}
I20260809 19:32:57.851686 24340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:57.851491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:46599)
0809 19:32:57.851605 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:57.851639 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260809 19:32:58.675287 24341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:58.675066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:56811)
0809 19:32:58.675191 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:58.675227 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260809 19:32:58.743467 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:58.743227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:49105)
0809 19:32:58.743351 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:58.743433 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260809 19:32:58.852257 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:58.852072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:50535)
0809 19:32:58.852186 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:58.852219 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260809 19:32:59.675925 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:59.675724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:38785)
0809 19:32:59.675842 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:59.675877 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260809 19:32:59.744031 24348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:59.743853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:36797)
0809 19:32:59.743972 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:59.744001 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260809 19:32:59.852861 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:32:59.852629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:54179)
0809 19:32:59.852789 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:32:59.852821 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260809 19:33:00.672768 24350 raft_consensus.cc:482] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:00.672879 24350 raft_consensus.cc:504] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df1564d18c0b49c297dd54f167a5b417" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 35801 } } peers { permanent_uuid: "95b1a5524afa4167850e7240cec576b8" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 45553 } } peers { permanent_uuid: "2b498c1490d54e38be13d50d5eccffa6" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36763 } }
I20260809 19:33:00.673051 24350 leader_election.cc:288] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801), 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553)
W20260809 19:33:00.673421 23717 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95b1a5524afa4167850e7240cec576b8 (127.31.57.194:45553): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
W20260809 19:33:00.673476 23714 leader_election.cc:341] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df1564d18c0b49c297dd54f167a5b417 (127.31.57.195:35801): Not found: Tablet not found: 5ecaef65d1304c53bda4590fe6053c08
I20260809 19:33:00.673504 23717 leader_election.cc:302] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b498c1490d54e38be13d50d5eccffa6; no voters: 95b1a5524afa4167850e7240cec576b8, df1564d18c0b49c297dd54f167a5b417
I20260809 19:33:00.673576 24350 raft_consensus.cc:2726] T 5ecaef65d1304c53bda4590fe6053c08 P 2b498c1490d54e38be13d50d5eccffa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:00.676476 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:00.676310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:53035)
0809 19:33:00.676409 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:00.676440 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260809 19:33:00.744604 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:00.744415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:42581)
0809 19:33:00.744536 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:00.744569 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260809 19:33:00.853571 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:00.853374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:33887)
0809 19:33:00.853500 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:00.853534 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260809 19:33:01.677106 24354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:01.676894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:48559)
0809 19:33:01.677013 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:01.677048 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 19:33:01.745173 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:01.745004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:36811)
0809 19:33:01.745110 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:01.745141 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260809 19:33:01.854199 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:01.853983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:58303)
0809 19:33:01.854138 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:01.854168 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260809 19:33:02.677842 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:02.677577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:43709)
0809 19:33:02.677711 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:02.677778 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260809 19:33:02.745798 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:02.745587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:34005)
0809 19:33:02.745718 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:02.745752 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260809 19:33:02.854949 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:02.854709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:39851)
0809 19:33:02.854873 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:02.854909 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260809 19:33:03.678665 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:03.678454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.193:45499)
0809 19:33:03.678588 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:03.678621 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260809 19:33:03.746447 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:03.746245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.194:57669)
0809 19:33:03.746379 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:03.746409 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260809 19:33:03.855816 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:03.855626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34331 (local address 127.31.57.195:41673)
0809 19:33:03.855741 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:03.855772 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:34331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260809 19:33:03.859584 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 23650
I20260809 19:33:03.874375 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 23887
I20260809 19:33:03.882848 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 24090
2026-08-09T19:33:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18623 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:33:03 dist-test-slave-x4qh krb5kdc[24371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:33:03 dist-test-slave-x4qh krb5kdc[24371](info): set up 2 sockets
Aug 09 19:33:03 dist-test-slave-x4qh krb5kdc[24371](info): commencing operation
krb5kdc: starting...
W20260809 19:33:05.063341 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:33:05.174185 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:05.248898 24390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:05.249099 24390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:05.249132 24390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:05.250244 24390 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:33:05.250309 24390 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:33:05.250330 24390 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:33:05.250348 24390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:33:05.250381 24390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:05.250407 24390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:05.250433 24390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:33:05.250463 24390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:33:05.251899 24390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:05.252152 24390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:05.252382 24390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:05.256989 24396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.256999 24402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.257103 24390 server_base.cc:1034] running on GCE node
W20260809 19:33:05.257103 24395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.257442 24390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:05.257797 24390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:05.258965 24390 hybrid_clock.cc:648] HybridClock initialized: now 1786303985258957 us; error 35 us; skew 500 ppm
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:05.264376 24390 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:33:05.264727 24390 webserver.cc:460] Webserver started at http://127.31.57.254:46045/ using document root <none> and password file <none>
I20260809 19:33:05.264917 24390 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:05.264956 24390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:05.265039 24390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:33:05.265960 24390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "05cb3d76d62742249b9bee44f62145ae"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "bb4e9c47e7c03486e74878f46b609dd5"
server_key_iv: "b338847b990f5e938bfea7cb7df1c19c"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.266252 24390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "05cb3d76d62742249b9bee44f62145ae"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "bb4e9c47e7c03486e74878f46b609dd5"
server_key_iv: "b338847b990f5e938bfea7cb7df1c19c"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.267500 24390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:33:05.268203 24416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.268364 24390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:33:05.268448 24390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "05cb3d76d62742249b9bee44f62145ae"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "bb4e9c47e7c03486e74878f46b609dd5"
server_key_iv: "b338847b990f5e938bfea7cb7df1c19c"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.268498 24390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:05.280690 24390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:05.282239 24390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:05.282337 24390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:05.286079 24390 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:33311
I20260809 19:33:05.286362 24487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:33311 every 8 connection(s)
I20260809 19:33:05.286427 24390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:33:05.286984 24488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:05.288213 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 24390
I20260809 19:33:05.288303 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:33:05.288488 31975 external_mini_cluster.cc:1428] Setting key 9164b66dcdea1eaccd6252de414ab7ff
I20260809 19:33:05.289909 24488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap starting.
I20260809 19:33:05.290669 24488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:05.290989 24488 log.cc:824] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:05.291692 24488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: No bootstrap required, opened a new log
I20260809 19:33:05.294025 24488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:05.294210 24488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:05.294247 24488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Initialized, Role: FOLLOWER
I20260809 19:33:05.294386 24488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:05.294457 24488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:33:05.294484 24488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:33:05.294534 24488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:05.295068 24488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:05.295151 24488 leader_election.cc:302] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05cb3d76d62742249b9bee44f62145ae; no voters:
I20260809 19:33:05.295356 24488 leader_election.cc:288] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:33:05.295441 24494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:33:05.295568 24494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 LEADER]: Becoming Leader. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Running, Role: LEADER
I20260809 19:33:05.295704 24494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:05.295704 24488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:33:05.296156 24495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:05.296242 24495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:05.296253 24497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05cb3d76d62742249b9bee44f62145ae. Latest consensus state: current_term: 1 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:05.296341 24497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:05.296684 24500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:33:05.297459 24500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:33:05.298887 24500 catalog_manager.cc:1347] Generated new cluster ID: 53cf4f86e8604c14942c7868ceb8bb2a
I20260809 19:33:05.298939 24500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:33:05.299649 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.288998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43676 (local address 127.31.57.254:33311)
0809 19:33:05.289154 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.289161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.289193 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:33:05.289735 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.289742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.289805 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.289908 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:05.290694 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.291667 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.292627 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.293043 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.293120 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.295718 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:05.295722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:05.295723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:05.295734 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:05.297972 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.298381 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.298384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.298385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.298479 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.298720 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.298722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.298722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.299041 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:05.299223 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.299335 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.299489 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260809 19:33:05.307363 24500 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:33:05.307896 24500 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:33:05.313048 24500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Generated new TSK 0
I20260809 19:33:05.313220 24500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:33:05.343623 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:05.425235 24532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:05.425434 24532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:05.425457 24532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:05.426543 24532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:05.426586 24532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:05.426611 24532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:33:05.428041 24532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:05.428273 24532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:05.428509 24532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:05.433377 24542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.433495 24544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.433640 24532 server_base.cc:1034] running on GCE node
W20260809 19:33:05.433403 24541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.433902 24532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:05.434185 24532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:05.435334 24532 hybrid_clock.cc:648] HybridClock initialized: now 1786303985435315 us; error 37 us; skew 500 ppm
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:05.437822 24532 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:33:05.438325 24532 webserver.cc:460] Webserver started at http://127.31.57.193:39013/ using document root <none> and password file <none>
I20260809 19:33:05.438604 24532 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:05.438671 24532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:05.438815 24532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:33:05.439899 24532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "b36edeb0c7c245d9b784a377c65a9d58"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "1a062e0cb76c625abc3c2c919b7ad840"
server_key_iv: "95e6b4c15e27232ccb596f042b73b7c5"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.440263 24532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "b36edeb0c7c245d9b784a377c65a9d58"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "1a062e0cb76c625abc3c2c919b7ad840"
server_key_iv: "95e6b4c15e27232ccb596f042b73b7c5"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.441509 24532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:33:05.442224 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.442418 24532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:33:05.442514 24532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "b36edeb0c7c245d9b784a377c65a9d58"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "1a062e0cb76c625abc3c2c919b7ad840"
server_key_iv: "95e6b4c15e27232ccb596f042b73b7c5"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.442595 24532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:05.451653 24532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:05.453397 24532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:05.453558 24532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:05.453789 24532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:05.454095 24532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:05.454152 24532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.454192 24532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:05.454223 24532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.460450 24532 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:33503
I20260809 19:33:05.460520 24728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:33503 every 8 connection(s)
I20260809 19:33:05.460834 24532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:05.468791 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 24532
I20260809 19:33:05.468873 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:33:05.469113 31975 external_mini_cluster.cc:1428] Setting key 302c04269d464870961606bbb150f26a
I20260809 19:33:05.469880 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.461554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:44611 (local address 127.31.57.254:33311)
0809 19:33:05.461614 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.461616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.461805 (+ 189us) server_negotiation.cc:407] Connection header received
0809 19:33:05.462331 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.462333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.462345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.462402 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:05.463347 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.463888 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.464914 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.465169 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.465259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.467149 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:05.467151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:05.467151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:05.467158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:05.468333 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.468755 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.468757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.468757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.468819 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.469126 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.469127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.469127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.469284 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:05.469368 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.469680 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.469779 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:33:05.470016 24731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.461530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:44611)
0809 19:33:05.461689 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.461722 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.462219 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.462466 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.462472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.463061 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:05.463239 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.463243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.464002 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.464003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.464819 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.464821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.465272 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.465273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.465410 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.465805 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:05.465812 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:05.467049 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:05.468415 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.468423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.468424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.468673 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.468920 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.468921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.468921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.469056 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.469359 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:05.469367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:05.469529 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.469626 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.469795 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 19:33:05.470630 24729 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:05.470737 24729 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:05.470918 24729 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:05.471606 24439 ts_manager.cc:194] Registered new tserver with Master: b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:05.472214 24439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:44611
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:33:05.492295 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:05.573848 24735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:05.574014 24735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:05.574035 24735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:05.574944 24735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:05.574976 24735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:05.574998 24735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:33:05.576246 24735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:05.576406 24735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:05.576577 24735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:05.581152 24740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.581161 24744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.581168 24741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.581537 24735 server_base.cc:1034] running on GCE node
I20260809 19:33:05.581727 24735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:05.581972 24735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:05.583120 24735 hybrid_clock.cc:648] HybridClock initialized: now 1786303985583096 us; error 38 us; skew 500 ppm
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:05.585588 24735 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:33:05.585988 24735 webserver.cc:460] Webserver started at http://127.31.57.194:34371/ using document root <none> and password file <none>
I20260809 19:33:05.586191 24735 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:05.586232 24735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:05.586447 24735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:33:05.587528 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "68a490665bd7875af942ebc20c4e42bb"
server_key_iv: "9291118ba6b1dea93c62069a13de0061"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.587881 24735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "68a490665bd7875af942ebc20c4e42bb"
server_key_iv: "9291118ba6b1dea93c62069a13de0061"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.589107 24735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:33:05.589778 24754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.589944 24735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:33:05.590075 24735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "68a490665bd7875af942ebc20c4e42bb"
server_key_iv: "9291118ba6b1dea93c62069a13de0061"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.590161 24735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:05.615336 24735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:05.617039 24735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:05.617197 24735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:05.617412 24735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:05.617717 24735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:05.617770 24735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.617799 24735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:05.617827 24735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.623267 24735 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:39185
I20260809 19:33:05.623315 24930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:39185 every 8 connection(s)
I20260809 19:33:05.623606 24735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:33:05.627802 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 24735
I20260809 19:33:05.627902 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:33:05.628103 31975 external_mini_cluster.cc:1428] Setting key 428eba4c71fdad70d368c1e826646891
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:05.632422 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.624211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:48883)
0809 19:33:05.624375 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.624442 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.624842 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.625025 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.625028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.625516 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:05.625679 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.625683 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.626238 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.626238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.626966 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.626968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.627294 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.627295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.627406 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.627825 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:05.627834 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:05.629382 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:05.630993 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.631000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.631002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.631339 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.631562 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.631563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.631563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.631709 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.631964 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:05.631969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:05.632088 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.632164 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.632289 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260809 19:33:05.632441 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.624240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:48883 (local address 127.31.57.254:33311)
0809 19:33:05.624308 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.624315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.624521 (+ 206us) server_negotiation.cc:407] Connection header received
0809 19:33:05.624917 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.624918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.624926 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.624979 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:05.625744 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.626164 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.627045 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.627218 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.627266 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.629463 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:05.629464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:05.629465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:05.629470 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:05.630914 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.631443 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.631444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.631445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.631499 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.631806 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.631806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.631807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.631910 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:05.631955 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.632234 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.632315 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:33:05.632929 24932 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:05.633023 24932 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:05.633181 24932 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:05.633565 24439 ts_manager.cc:194] Registered new tserver with Master: ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
I20260809 19:33:05.633889 24439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:48883
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:33:05.654013 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:05.731273 24938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:05.731446 24938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:05.731467 24938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:05.732424 24938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:05.732461 24938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:05.732486 24938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:33:05.733783 24938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:05.733949 24938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:05.734099 24938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:05.738595 24949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.738616 24943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:05.738603 24945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:05.738699 24938 server_base.cc:1034] running on GCE node
I20260809 19:33:05.738969 24938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:05.739197 24938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:05.740340 24938 hybrid_clock.cc:648] HybridClock initialized: now 1786303985740331 us; error 25 us; skew 500 ppm
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:05.742832 24938 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:33:05.743257 24938 webserver.cc:460] Webserver started at http://127.31.57.195:36465/ using document root <none> and password file <none>
I20260809 19:33:05.743489 24938 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:05.743532 24938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:05.743664 24938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:33:05.744554 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "6643af3689da3783cfe497b0889c03cf"
server_key_iv: "13afdd61a08a8d40ca1b69bef50d24b8"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.744874 24938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "6643af3689da3783cfe497b0889c03cf"
server_key_iv: "13afdd61a08a8d40ca1b69bef50d24b8"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.746206 24938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:33:05.746912 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.747100 24938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:33:05.747185 24938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "6643af3689da3783cfe497b0889c03cf"
server_key_iv: "13afdd61a08a8d40ca1b69bef50d24b8"
server_key_version: "encryptionkey@0"
I20260809 19:33:05.747253 24938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:05.753334 24938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:05.754916 24938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:05.755051 24938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:05.755250 24938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:05.755591 24938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:05.755645 24938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.755694 24938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:05.755718 24938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:05.761269 24938 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:32799
I20260809 19:33:05.761317 25118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:32799 every 8 connection(s)
I20260809 19:33:05.761584 24938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:33:05 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303985, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:05.769476 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 24938
I20260809 19:33:05.769582 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:33:05.769780 31975 external_mini_cluster.cc:1428] Setting key 4c69851ca3f01da9e5cebd9aa2b629e5
I20260809 19:33:05.770041 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.762230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:52557 (local address 127.31.57.254:33311)
0809 19:33:05.762323 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.762325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.762491 (+ 166us) server_negotiation.cc:407] Connection header received
0809 19:33:05.762949 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.762951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.762963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.763012 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:05.763895 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.764375 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.765262 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.765488 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.765583 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.767587 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:05.767589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:05.767589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:05.767596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:05.768827 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.769156 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.769157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.769157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.769202 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.769410 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.769412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.769412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.769524 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:05.769597 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.769879 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.769958 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:33:05.770167 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.762188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:52557)
0809 19:33:05.762368 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.762420 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.762840 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.763070 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.763080 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.763637 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:05.763795 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.763799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.764475 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.764476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.765172 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.765174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.765594 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.765595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.765703 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.766189 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:05.766196 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:05.767508 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:05.768896 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.768901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.768903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.769089 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.769255 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:05.769256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:05.769256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:05.769341 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:05.769590 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:05.769596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:05.769742 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.769842 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.769977 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260809 19:33:05.770597 25120 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:05.770709 25120 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:05.770844 25120 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:05.771162 24439 ts_manager.cc:194] Registered new tserver with Master: b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799)
I20260809 19:33:05.771561 24439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:52557
I20260809 19:33:05.781458 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:33:05.788125 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.782788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.31.57.254:33311)
0809 19:33:05.782850 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.782852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.782962 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:33:05.783013 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.783014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.783025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.783089 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:05.783657 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.784099 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.784763 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.784928 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.784974 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.786065 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:05.786066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:05.786066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:05.786070 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:05.787203 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.787553 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.787554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.787555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.787601 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:05.787728 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:05.787728 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:05.787728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:05.787825 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:05.787888 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.787987 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.788037 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:33:05.789561 24439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:33:05.791105 24439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:33:05.797087 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39185 (local address 127.0.0.1:57778)
0809 19:33:05.793445 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.793455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.793516 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.793707 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.793710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.793713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:05.793768 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.793770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.794724 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.794725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.795800 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.795802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.796743 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.796769 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.796863 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.796878 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.796949 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.796999 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":65,"threads_started":1}
I20260809 19:33:05.797355 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57778 (local address 127.31.57.194:39185)
0809 19:33:05.793383 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.793389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.793505 (+ 116us) server_negotiation.cc:407] Connection header received
0809 19:33:05.793582 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.793584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.793597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.793697 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:05.793826 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.794621 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.795946 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.796650 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.796738 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.796789 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.797201 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.797255 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":101,"threads_started":1}
I20260809 19:33:05.797435 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:32799 (local address 127.0.0.1:59244)
0809 19:33:05.793372 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.793410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.793481 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.793784 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.793785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.793788 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:05.793839 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.793841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.795135 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.795136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.796180 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.796183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.797230 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.797231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.797288 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.797293 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.797338 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.797365 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":54,"threads_started":1}
I20260809 19:33:05.797437 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.0.0.1:58574)
0809 19:33:05.793543 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.793553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.793594 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.793932 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.793933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.793935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:05.793978 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.793980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.795171 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.795172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.796148 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.796151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.797093 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.797094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.797253 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.797257 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.797312 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.797356 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":71,"threads_started":1}
I20260809 19:33:05.797784 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.31.57.193:33503)
0809 19:33:05.793708 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.793713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.793721 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:33:05.793758 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.793760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.793787 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.793857 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:05.794068 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.795078 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.796236 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.797014 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.797078 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.797301 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.797332 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.797377 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":95,"threads_started":1}
I20260809 19:33:05.798085 24823 tablet_service.cc:1467] Processing CreateTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc (DEFAULT_TABLE table=test-table [id=14902cda400d4043a03479285ae5758b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:05.798481 24823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52f0dee9a7f4bbba4039166ef2578fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:05.798449 24620 tablet_service.cc:1467] Processing CreateTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc (DEFAULT_TABLE table=test-table [id=14902cda400d4043a03479285ae5758b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:05.798738 24620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52f0dee9a7f4bbba4039166ef2578fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:05.799614 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.793080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.31.57.195:32799)
0809 19:33:05.793451 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.793455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.793466 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:33:05.793640 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.793644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.793659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.793728 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:05.794026 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.795028 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.796260 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.797085 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.797149 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.797200 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.799506 (+ 2306us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.799548 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":86,"threads_started":1}
I20260809 19:33:05.800146 25026 tablet_service.cc:1467] Processing CreateTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc (DEFAULT_TABLE table=test-table [id=14902cda400d4043a03479285ae5758b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:05.800446 25026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52f0dee9a7f4bbba4039166ef2578fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:05.801566 25145 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap starting.
I20260809 19:33:05.801813 25147 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Bootstrap starting.
I20260809 19:33:05.802353 25145 tablet_bootstrap.cc:654] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:05.802413 25147 tablet_bootstrap.cc:654] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:05.802985 25148 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Bootstrap starting.
I20260809 19:33:05.803030 25147 log.cc:824] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:05.803033 25145 log.cc:824] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:05.803678 25148 tablet_bootstrap.cc:654] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:05.803948 25145 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: No bootstrap required, opened a new log
I20260809 19:33:05.804030 25145 ts_tablet_manager.cc:1397] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:33:05.804180 25148 log.cc:824] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:05.804872 25148 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: No bootstrap required, opened a new log
I20260809 19:33:05.804922 25148 ts_tablet_manager.cc:1397] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:33:05.805616 25145 raft_consensus.cc:348] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.805775 25145 raft_consensus.cc:374] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:05.805823 25145 raft_consensus.cc:729] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36edeb0c7c245d9b784a377c65a9d58, State: Initialized, Role: FOLLOWER
I20260809 19:33:05.805943 25145 consensus_queue.cc:260] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.806216 25145 ts_tablet_manager.cc:1428] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:33:05.806260 24729 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
I20260809 19:33:05.806393 25148 raft_consensus.cc:348] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.806591 25148 raft_consensus.cc:374] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:05.806638 25148 raft_consensus.cc:729] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e4a45a478d49dcb79a379773bdcb0b, State: Initialized, Role: FOLLOWER
I20260809 19:33:05.806756 25147 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: No bootstrap required, opened a new log
I20260809 19:33:05.806754 25148 consensus_queue.cc:260] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.806826 25147 ts_tablet_manager.cc:1397] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 19:33:05.807015 25148 ts_tablet_manager.cc:1428] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:33:05.807123 25120 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
I20260809 19:33:05.808588 25147 raft_consensus.cc:348] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.808708 25147 raft_consensus.cc:374] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:05.808727 25147 raft_consensus.cc:729] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3d2d60e71343fcaee9e5a7a65d5d31, State: Initialized, Role: FOLLOWER
I20260809 19:33:05.808851 25147 consensus_queue.cc:260] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.809105 25147 ts_tablet_manager.cc:1428] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:33:05.809137 24932 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
W20260809 19:33:05.874185 24933 tablet.cc:2369] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:33:05.942420 25160 raft_consensus.cc:482] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:05.942534 25160 raft_consensus.cc:504] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.942895 25160 leader_election.cc:288] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:05.945762 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.943062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:32799 (local address 127.31.57.194:51935)
0809 19:33:05.943132 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.943145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.943189 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.943329 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.943331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.943333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:05.943429 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.943436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.944082 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.944083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.944907 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.944909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.945496 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.945497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.945578 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.945580 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.945648 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.945688 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 19:33:05.945806 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.943113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:51935 (local address 127.31.57.195:32799)
0809 19:33:05.943183 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.943199 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.943212 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:33:05.943246 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.943247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.943257 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.943321 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:05.943519 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.943996 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.944993 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.945412 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.945499 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.945508 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.945654 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.945688 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:33:05.946120 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.943212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.31.57.194:50145)
0809 19:33:05.943415 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:05.943424 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:05.943478 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:05.943618 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:05.943619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:05.943621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:05.943696 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.943697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.944374 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.944375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.945160 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:05.945162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.945836 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.945837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:05.945915 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.945919 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:33:05.945994 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:33:05.946043 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":113,"threads_started":1}
I20260809 19:33:05.946169 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.943322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:50145 (local address 127.31.57.193:33503)
0809 19:33:05.943393 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.943394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.943465 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:33:05.943531 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.943532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.943540 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.943614 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:05.943781 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.944247 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.945289 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.945729 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.945817 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.945845 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.946059 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.946095 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:33:05.946307 25055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52f0dee9a7f4bbba4039166ef2578fc" candidate_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" is_pre_election: true
I20260809 19:33:05.946496 25055 raft_consensus.cc:2454] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed3d2d60e71343fcaee9e5a7a65d5d31 in term 0.
I20260809 19:33:05.946476 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52f0dee9a7f4bbba4039166ef2578fc" candidate_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36edeb0c7c245d9b784a377c65a9d58" is_pre_election: true
I20260809 19:33:05.946642 24647 raft_consensus.cc:2454] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed3d2d60e71343fcaee9e5a7a65d5d31 in term 0.
I20260809 19:33:05.946767 24756 leader_election.cc:302] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31; no voters:
I20260809 19:33:05.946906 25160 raft_consensus.cc:2781] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:33:05.946987 25160 raft_consensus.cc:482] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:33:05.947026 25160 raft_consensus.cc:3037] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:05.947811 25160 raft_consensus.cc:504] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.947965 25160 leader_election.cc:288] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [CANDIDATE]: Term 1 election: Requested vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:05.948128 25055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52f0dee9a7f4bbba4039166ef2578fc" candidate_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
I20260809 19:33:05.948184 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52f0dee9a7f4bbba4039166ef2578fc" candidate_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36edeb0c7c245d9b784a377c65a9d58"
I20260809 19:33:05.948210 25055 raft_consensus.cc:3037] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:05.948283 24647 raft_consensus.cc:3037] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:05.948921 25055 raft_consensus.cc:2454] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3d2d60e71343fcaee9e5a7a65d5d31 in term 1.
I20260809 19:33:05.949023 24647 raft_consensus.cc:2454] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3d2d60e71343fcaee9e5a7a65d5d31 in term 1.
I20260809 19:33:05.949107 24756 leader_election.cc:302] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31; no voters:
I20260809 19:33:05.949232 25160 raft_consensus.cc:2781] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:33:05.949465 25160 raft_consensus.cc:686] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 LEADER]: Becoming Leader. State: Replica: ed3d2d60e71343fcaee9e5a7a65d5d31, State: Running, Role: LEADER
I20260809 19:33:05.949568 25160 consensus_queue.cc:237] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:05.950361 24439 catalog_manager.cc:5495] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194). New cstate: current_term: 1 leader_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } health_report { overall_health: HEALTHY } } }
W20260809 19:33:05.961577 24730 tablet.cc:2369] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:33:05.978374 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:05.976234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.31.57.194:39185)
0809 19:33:05.976297 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:05.976298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:05.976308 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:33:05.976339 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:05.976340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:05.976361 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:05.976420 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:33:05.976568 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.977026 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.977728 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:05.977891 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:05.977937 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:05.978119 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:33:05.978203 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:33:05.978257 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:05.978299 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 19:33:05.978315 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:33:05.980014 24647 raft_consensus.cc:1261] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Refusing update from remote peer ed3d2d60e71343fcaee9e5a7a65d5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:33:05.980026 25055 raft_consensus.cc:1261] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Refusing update from remote peer ed3d2d60e71343fcaee9e5a7a65d5d31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:33:05.980382 25160 consensus_queue.cc:1035] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:33:05.980528 25162 consensus_queue.cc:1035] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:33:05.983209 25170 mvcc.cc:205] Tried to move back new op lower bound from 7316701126571704320 to 7316701126449831936. Current Snapshot: MvccSnapshot[applied={T|T < 7316701126571704320}]
I20260809 19:33:05.983857 25167 mvcc.cc:205] Tried to move back new op lower bound from 7316701126571704320 to 7316701126449831936. Current Snapshot: MvccSnapshot[applied={T|T < 7316701126571704320}]
I20260809 19:33:05.985119 24436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:33:05.985230 24436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:33:05.986027 24436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Sending DeleteTablet for 3 replicas of tablet c52f0dee9a7f4bbba4039166ef2578fc
I20260809 19:33:05.986367 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 24532
I20260809 19:33:05.986403 24620 tablet_service.cc:1514] Processing DeleteTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:05 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:58574
I20260809 19:33:05.986413 24823 tablet_service.cc:1514] Processing DeleteTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:05 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:57778
I20260809 19:33:05.991442 25026 tablet_service.cc:1514] Processing DeleteTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:05 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:59244
I20260809 19:33:05.992102 25183 tablet_replica.cc:330] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: stopping tablet replica
I20260809 19:33:05.992218 25181 tablet_replica.cc:330] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: stopping tablet replica
I20260809 19:33:05.992338 25181 raft_consensus.cc:2229] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:33:05.992443 25183 raft_consensus.cc:2229] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:33:05.992558 25183 raft_consensus.cc:2258] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:33:05.992664 25181 raft_consensus.cc:2258] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:33:05.993431 25181 ts_tablet_manager.cc:1905] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:33:05.995414 25183 ts_tablet_manager.cc:1905] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:33:05.998644 25181 ts_tablet_manager.cc:1918] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:33:05.999660 25183 ts_tablet_manager.cc:1918] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:33:05.999775 25181 log.cc:1196] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/c52f0dee9a7f4bbba4039166ef2578fc
I20260809 19:33:06.000068 25181 ts_tablet_manager.cc:1939] T c52f0dee9a7f4bbba4039166ef2578fc P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting consensus metadata
I20260809 19:33:06.000200 25183 log.cc:1196] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/c52f0dee9a7f4bbba4039166ef2578fc
I20260809 19:33:06.000494 25183 ts_tablet_manager.cc:1939] T c52f0dee9a7f4bbba4039166ef2578fc P b6e4a45a478d49dcb79a379773bdcb0b: Deleting consensus metadata
I20260809 19:33:06.000927 24419 catalog_manager.cc:4841] TS b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): tablet c52f0dee9a7f4bbba4039166ef2578fc (table test-table [id=14902cda400d4043a03479285ae5758b]) successfully deleted
I20260809 19:33:06.001175 24417 catalog_manager.cc:4841] TS ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): tablet c52f0dee9a7f4bbba4039166ef2578fc (table test-table [id=14902cda400d4043a03479285ae5758b]) successfully deleted
W20260809 19:33:06.005147 24419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:33503 (error 108)
I20260809 19:33:06.005389 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 24735
I20260809 19:33:06.012907 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:06.005506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.0.0.1:58590)
0809 19:33:06.012841 (+ 7335us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:06.012882 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7315}
W20260809 19:33:06.015357 24419 catalog_manager.cc:4584] TS b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c52f0dee9a7f4bbba4039166ef2578fc: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33503: connect: Connection refused (error 111)
I20260809 19:33:06.020983 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 24938
I20260809 19:33:06.036550 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 24390
I20260809 19:33:26.046002 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--webserver_interface=127.31.57.254
--webserver_port=46045
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:26.127341 25185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:26.127522 25185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:26.127542 25185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:26.128680 25185 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:33:26.128720 25185 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:33:26.128733 25185 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:33:26.128746 25185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:33:26.128794 25185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:26.128813 25185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:26.128835 25185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:33:26.128865 25185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:33:26.130218 25185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=46045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:26.130379 25185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:26.130592 25185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:26.139622 25195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.139623 25192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.139647 25193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:26.139699 25185 server_base.cc:1034] running on GCE node
I20260809 19:33:26.140120 25185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:26.140369 25185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:26.141510 25185 hybrid_clock.cc:648] HybridClock initialized: now 1786304006141496 us; error 29 us; skew 500 ppm
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:26.148423 25185 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:33:26.148900 25185 webserver.cc:460] Webserver started at http://127.31.57.254:46045/ using document root <none> and password file <none>
I20260809 19:33:26.149134 25185 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:26.149209 25185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:26.150502 25185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:26.151170 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.151340 25185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:33:26.151464 25185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "05cb3d76d62742249b9bee44f62145ae"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "bb4e9c47e7c03486e74878f46b609dd5"
server_key_iv: "b338847b990f5e938bfea7cb7df1c19c"
server_key_version: "encryptionkey@0"
I20260809 19:33:26.151820 25185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:26.162428 25185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:26.164691 25185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:26.164849 25185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:26.168879 25185 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:33311
I20260809 19:33:26.168910 25287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:33311 every 8 connection(s)
I20260809 19:33:26.169268 25185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:33:26.171128 25289 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:33:26.171797 25289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap starting.
I20260809 19:33:26.173198 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 25185
I20260809 19:33:26.173552 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:33503
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=39013
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:33:26.181730 25289 log.cc:824] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:26.183859 25289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:33:26.184204 25289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap complete.
I20260809 19:33:26.186302 25289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:26.186501 25289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Initialized, Role: FOLLOWER
I20260809 19:33:26.186652 25289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:26.186724 25289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:33:26.186775 25289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:33:26.186829 25289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:33:26.187927 25289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:26.188016 25289 leader_election.cc:302] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05cb3d76d62742249b9bee44f62145ae; no voters:
I20260809 19:33:26.188241 25289 leader_election.cc:288] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:33:26.188279 25293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:33:26.188457 25293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 LEADER]: Becoming Leader. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Running, Role: LEADER
I20260809 19:33:26.188513 25289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:33:26.188617 25293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:26.189005 25298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05cb3d76d62742249b9bee44f62145ae. Latest consensus state: current_term: 2 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:26.189090 25298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:26.189378 25297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:26.189429 25303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:33:26.189450 25297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:26.190183 25303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:33:26.190284 25303 catalog_manager.cc:1259] Loaded cluster ID: 53cf4f86e8604c14942c7868ceb8bb2a
I20260809 19:33:26.190325 25303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:33:26.191443 25303 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:33:26.191617 25303 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Loaded TSK: 0
I20260809 19:33:26.199656 25303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Generated new TSK 1
I20260809 19:33:26.199740 25303 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:33:26.254508 25291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:26.254700 25291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:26.254737 25291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:26.255997 25291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:26.256057 25291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:26.256102 25291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:33:26.257493 25291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:33503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=39013
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:26.257764 25291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:26.257992 25291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:26.263576 25328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.263624 25333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.263865 25327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:26.264394 25291 server_base.cc:1034] running on GCE node
I20260809 19:33:26.264616 25291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:26.264873 25291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:26.266042 25291 hybrid_clock.cc:648] HybridClock initialized: now 1786304006266030 us; error 32 us; skew 500 ppm
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:26.268703 25291 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:33:26.269162 25291 webserver.cc:460] Webserver started at http://127.31.57.193:39013/ using document root <none> and password file <none>
I20260809 19:33:26.269402 25291 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:26.269486 25291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:26.270826 25291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:26.271582 25345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.271772 25291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:26.271870 25291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "b36edeb0c7c245d9b784a377c65a9d58"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "1a062e0cb76c625abc3c2c919b7ad840"
server_key_iv: "95e6b4c15e27232ccb596f042b73b7c5"
server_key_version: "encryptionkey@0"
I20260809 19:33:26.272195 25291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:26.279894 25291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:26.281867 25291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:26.282012 25291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:26.282229 25291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:26.282676 25355 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:33:26.284495 25291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:33:26.284554 25291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:33:26.284596 25291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:33:26.285163 25291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:33:26.285218 25291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:26.285283 25355 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap starting.
I20260809 19:33:26.293017 25291 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:33503
I20260809 19:33:26.293062 25520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:33503 every 8 connection(s)
I20260809 19:33:26.293504 25291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:33:26.296766 25355 log.cc:824] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:26.298878 25355 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:33:26.299221 25355 tablet_bootstrap.cc:492] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap complete.
I20260809 19:33:26.300550 25355 ts_tablet_manager.cc:1397] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260809 19:33:26.301599 25355 raft_consensus.cc:348] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:26.301913 25355 raft_consensus.cc:729] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36edeb0c7c245d9b784a377c65a9d58, State: Initialized, Role: FOLLOWER
I20260809 19:33:26.302052 25355 consensus_queue.cc:260] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } }
I20260809 19:33:26.302348 25355 ts_tablet_manager.cc:1428] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:26.303406 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 25291
I20260809 19:33:26.303678 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:39185
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=34371
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:33:26.307611 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.294391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:49945)
0809 19:33:26.295645 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.295685 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.296222 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.296810 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.296814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.297106 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:26.297360 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.297366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.299078 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.299080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.300505 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.300508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.301309 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.301311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.301456 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.302113 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:26.302138 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:26.303864 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:26.305922 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.305929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.305930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.306135 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.306472 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.306473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.306474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.306575 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.306987 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:26.306994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:26.307128 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.307224 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.307423 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":100,"threads_started":1}
I20260809 19:33:26.307623 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.294467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:49945 (local address 127.31.57.254:33311)
0809 19:33:26.296413 (+ 1946us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.296421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.296453 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:33:26.296568 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.296574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.296620 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.296749 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.297500 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.298964 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.300607 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.301184 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.301293 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.303966 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.303969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.303970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.303980 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.305837 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.306248 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.306251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.306251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.306392 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.306677 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.306678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.306678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.306894 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.306994 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.307295 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.307439 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1901,"thread_start_us":74,"threads_started":1}
I20260809 19:33:26.308393 25522 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:26.308501 25522 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:26.308676 25522 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:26.309461 25234 ts_manager.cc:194] Registered new tserver with Master: b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:26.310708 25234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:49945
I20260809 19:33:26.311785 25522 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.193@KRBTEST.COM
I20260809 19:33:26.318888 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.310069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.0.0.1:50692)
0809 19:33:26.310231 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.310261 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.310317 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.311349 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.311356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.311638 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:26.311693 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.311695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.312631 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.312632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.313114 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.313116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.313544 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.313545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.313633 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.314119 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:26.314123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:26.315711 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:26.317651 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.317658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.317659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.317916 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.318165 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.318166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.318167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.318227 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.318543 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:26.318550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:26.318599 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.318680 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.318784 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260809 19:33:26.318913 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.311042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.31.57.193:33503)
0809 19:33:26.311198 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.311202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.311211 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:33:26.311245 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.311247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.311258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.311345 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.311780 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.312560 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.313196 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.313460 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.313532 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.315816 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.315823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.315823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.315830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.317559 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.318010 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.318012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.318013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.318087 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.318317 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.318318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.318318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.318475 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.318552 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.318730 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.318821 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260809 19:33:26.319432 25417 tablet_service.cc:1514] Processing DeleteTablet for tablet c52f0dee9a7f4bbba4039166ef2578fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:05 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50692
I20260809 19:33:26.319914 25534 tablet_replica.cc:330] stopping tablet replica
I20260809 19:33:26.320003 25534 raft_consensus.cc:2229] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:33:26.320056 25534 raft_consensus.cc:2258] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:33:26.320346 25534 ts_tablet_manager.cc:1905] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:33:26.322314 25534 ts_tablet_manager.cc:1918] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:33:26.322400 25534 log.cc:1196] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal/wals/c52f0dee9a7f4bbba4039166ef2578fc
I20260809 19:33:26.322651 25534 ts_tablet_manager.cc:1939] T c52f0dee9a7f4bbba4039166ef2578fc P b36edeb0c7c245d9b784a377c65a9d58: Deleting consensus metadata
I20260809 19:33:26.323197 25209 catalog_manager.cc:4841] TS b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503): tablet c52f0dee9a7f4bbba4039166ef2578fc (table test-table [id=14902cda400d4043a03479285ae5758b]) successfully deleted
W20260809 19:33:26.385845 25531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:26.386006 25531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:26.386025 25531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:26.387002 25531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:26.387038 25531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:26.387061 25531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:33:26.388373 25531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=34371
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:26.388602 25531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:26.388823 25531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:26.393949 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.394008 25550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.393937 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:26.394057 25531 server_base.cc:1034] running on GCE node
I20260809 19:33:26.394356 25531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:26.394619 25531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:26.395772 25531 hybrid_clock.cc:648] HybridClock initialized: now 1786304006395750 us; error 39 us; skew 500 ppm
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:26.398046 25531 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:33:26.398469 25531 webserver.cc:460] Webserver started at http://127.31.57.194:34371/ using document root <none> and password file <none>
I20260809 19:33:26.398694 25531 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:26.398782 25531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:26.400084 25531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:26.400687 25566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.400837 25531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:33:26.400930 25531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "68a490665bd7875af942ebc20c4e42bb"
server_key_iv: "9291118ba6b1dea93c62069a13de0061"
server_key_version: "encryptionkey@0"
I20260809 19:33:26.401216 25531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:26.434962 25531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:26.436611 25531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:26.436771 25531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:26.437000 25531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:26.437299 25531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:26.437353 25531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.437390 25531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:26.437417 25531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.443235 25531 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:39185
I20260809 19:33:26.443271 25738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:39185 every 8 connection(s)
I20260809 19:33:26.443611 25531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:26.451159 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 25531
I20260809 19:33:26.451493 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:32799
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=36465
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:33:26.452665 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.444361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:49837)
0809 19:33:26.444521 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.444551 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.445006 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.445242 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.445249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.445784 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:26.445956 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.445960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.446814 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.446815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.447812 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.447815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.448201 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.448202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.448348 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.448735 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:26.448743 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:26.450031 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:26.451305 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.451312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.451314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.451611 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.451852 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.451853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.451853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.451943 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.452227 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:26.452231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:26.452330 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.452404 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.452507 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260809 19:33:26.452795 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.444377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49837 (local address 127.31.57.254:33311)
0809 19:33:26.444438 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.444440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.444625 (+ 185us) server_negotiation.cc:407] Connection header received
0809 19:33:26.445108 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.445109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.445124 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.445176 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.446073 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.446726 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.447899 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.448107 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.448189 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.450113 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.450114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.450114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.450120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.451199 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.451722 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.451723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.451724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.451784 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.452024 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.452025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.452025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.452163 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.452230 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.452448 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.452542 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:33:26.453459 25739 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:26.453534 25739 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:26.453737 25739 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:26.454105 25234 ts_manager.cc:194] Registered new tserver with Master: ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
I20260809 19:33:26.454609 25234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:49837
W20260809 19:33:26.528600 25743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:26.528800 25743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:26.528833 25743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:26.529925 25743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:26.529989 25743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:26.530035 25743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:33:26.531358 25743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:32799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=36465
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:26.531629 25743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:26.531831 25743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:26.536600 25751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:26.536659 25743 server_base.cc:1034] running on GCE node
W20260809 19:33:26.536621 25753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:26.536597 25750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:26.537014 25743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:26.537256 25743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:26.538416 25743 hybrid_clock.cc:648] HybridClock initialized: now 1786304006538426 us; error 59 us; skew 500 ppm
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:26.540802 25743 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:33:26.541229 25743 webserver.cc:460] Webserver started at http://127.31.57.195:36465/ using document root <none> and password file <none>
I20260809 19:33:26.541486 25743 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:26.541567 25743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:26.542824 25743 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:26.543360 25761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.543565 25743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:33:26.543660 25743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "6643af3689da3783cfe497b0889c03cf"
server_key_iv: "13afdd61a08a8d40ca1b69bef50d24b8"
server_key_version: "encryptionkey@0"
I20260809 19:33:26.543968 25743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:26.581981 25743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:26.583868 25743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:26.583982 25743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:26.584143 25743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:26.584451 25743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:26.584486 25743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.584506 25743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:26.584551 25743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:26.590765 25743 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:32799
I20260809 19:33:26.590843 25934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:32799 every 8 connection(s)
I20260809 19:33:26.591145 25743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:33:26 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:33:26.598316 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 25743
I20260809 19:33:26.599705 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.591887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:37921 (local address 127.31.57.254:33311)
0809 19:33:26.591941 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.591943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.592172 (+ 229us) server_negotiation.cc:407] Connection header received
0809 19:33:26.592627 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.592628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.592637 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.592684 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.593467 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.593979 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.594893 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.595056 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.595102 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.597014 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.597016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.597016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.597022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.598151 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.598600 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.598601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.598601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.598677 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.599006 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.599007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.599007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.599162 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.599222 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.599574 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.599644 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":24}
I20260809 19:33:26.600019 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.591876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:37921)
0809 19:33:26.592018 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.592076 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.592541 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.592732 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.592735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.593229 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:26.593395 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.593400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.594058 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.594058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.594821 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.594823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.595123 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.595124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.595222 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.595601 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:26.595608 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:26.596927 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:26.598227 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.598233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.598234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.598507 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.598761 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:26.598763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:26.598763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:26.598923 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:26.599236 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:26.599243 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:26.599433 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.599558 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.599723 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260809 19:33:26.600446 25935 heartbeater.cc:347] Connected to a master server at 127.31.57.254:33311
I20260809 19:33:26.600533 25935 heartbeater.cc:464] Registering TS with master...
I20260809 19:33:26.600692 25935 heartbeater.cc:511] Master 127.31.57.254:33311 requested a full tablet report, sending...
I20260809 19:33:26.601073 25234 ts_manager.cc:194] Registered new tserver with Master: b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799)
I20260809 19:33:26.601500 25234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:37921
I20260809 19:33:26.605172 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.598756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.31.57.254:33311)
0809 19:33:26.598896 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.598897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.598991 (+ 94us) server_negotiation.cc:407] Connection header received
0809 19:33:26.599040 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.599041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.599052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.599121 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.599927 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.600596 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.601573 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.601781 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.601863 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.602916 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.602917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.602917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.602922 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.604122 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.604479 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.604480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.604480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.604528 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.604772 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.604773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.604773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.604864 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.604975 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.605061 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.605117 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260809 19:33:26.605883 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:33:26.608259 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.606184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.31.57.254:33311)
0809 19:33:26.606213 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.606214 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.606342 (+ 128us) server_negotiation.cc:407] Connection header received
0809 19:33:26.606383 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.606384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.606407 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.606461 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:33:26.606524 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.606956 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.607723 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.607898 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.607950 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.608074 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:33:26.608108 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 19:33:26.608225 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39096: token expired
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:33:26.608328 25949 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39096: token expired
W20260809 19:33:26.608419 25130 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.57.254:33311)
I20260809 19:33:26.613297 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.608672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.31.57.254:33311)
0809 19:33:26.608701 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.608703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.608711 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:33:26.608749 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.608750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.608756 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.608805 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:26.609187 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.609627 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.610239 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.610408 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.610454 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.611569 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:26.611570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:26.611570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:26.611575 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:26.612571 (+ 996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.612838 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.612839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.612839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.612881 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:26.613015 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:26.613015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:26.613015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:26.613085 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:26.613140 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.613197 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.613245 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:33:26.614265 25234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:33:26.615698 25234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:33:26.617687 25417 tablet_service.cc:1467] Processing CreateTablet for tablet 1e073e977005455f9fe189d287c786f3 (DEFAULT_TABLE table=test-table [id=70f145bb3bdf4f558090fda740a1b91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:26.617893 25417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e073e977005455f9fe189d287c786f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:26.619561 25355 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap starting.
I20260809 19:33:26.619906 25355 tablet_bootstrap.cc:654] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:26.620450 25355 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: No bootstrap required, opened a new log
I20260809 19:33:26.620528 25355 ts_tablet_manager.cc:1397] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:26.620827 25355 raft_consensus.cc:348] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.620908 25355 raft_consensus.cc:374] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:26.620955 25355 raft_consensus.cc:729] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36edeb0c7c245d9b784a377c65a9d58, State: Initialized, Role: FOLLOWER
I20260809 19:33:26.620997 25355 consensus_queue.cc:260] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.621086 25355 ts_tablet_manager.cc:1428] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:26.621354 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.617352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:32799 (local address 127.0.0.1:60658)
0809 19:33:26.617399 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.617409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.617445 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.618026 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.618027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.618028 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:26.618070 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.618071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.619345 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.619346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.620358 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.620360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.621121 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.621122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.621192 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.621210 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.621241 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.621284 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 19:33:26.621363 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.617453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39185 (local address 127.0.0.1:44344)
0809 19:33:26.617575 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.617585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.617639 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.618036 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.618037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.618039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:26.618083 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.618084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.619085 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.619086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.620205 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.620208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.621171 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.621172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.621216 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.621219 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.621262 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.621312 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260809 19:33:26.621534 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.617605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.31.57.194:39185)
0809 19:33:26.617809 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.617813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.617824 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:33:26.617863 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.617865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.617880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.617992 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:26.618146 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.619004 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.620358 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.621026 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.621093 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.621145 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.621401 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.621447 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260809 19:33:26.621734 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.617592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.31.57.195:32799)
0809 19:33:26.617834 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.617838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.617849 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:33:26.617882 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.617884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.617898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.618140 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:26.618204 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.619245 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.620437 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.621025 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.621551 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.621610 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.621632 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.621663 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":71,"threads_started":1}
I20260809 19:33:26.622274 25630 tablet_service.cc:1467] Processing CreateTablet for tablet 1e073e977005455f9fe189d287c786f3 (DEFAULT_TABLE table=test-table [id=70f145bb3bdf4f558090fda740a1b91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:26.622397 25833 tablet_service.cc:1467] Processing CreateTablet for tablet 1e073e977005455f9fe189d287c786f3 (DEFAULT_TABLE table=test-table [id=70f145bb3bdf4f558090fda740a1b91a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:33:26.622668 25833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e073e977005455f9fe189d287c786f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:26.622666 25630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e073e977005455f9fe189d287c786f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:33:26.626047 25959 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Bootstrap starting.
I20260809 19:33:26.626252 25957 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Bootstrap starting.
I20260809 19:33:26.626719 25959 tablet_bootstrap.cc:654] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:26.626986 25957 tablet_bootstrap.cc:654] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Neither blocks nor log segments found. Creating new log.
I20260809 19:33:26.627411 25959 log.cc:824] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:26.627616 25957 log.cc:824] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:26.628289 25959 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: No bootstrap required, opened a new log
I20260809 19:33:26.628396 25959 ts_tablet_manager.cc:1397] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:33:26.628525 25957 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: No bootstrap required, opened a new log
I20260809 19:33:26.628580 25957 ts_tablet_manager.cc:1397] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:33:26.629729 25959 raft_consensus.cc:348] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.629868 25959 raft_consensus.cc:374] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:26.629910 25959 raft_consensus.cc:729] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3d2d60e71343fcaee9e5a7a65d5d31, State: Initialized, Role: FOLLOWER
I20260809 19:33:26.629861 25957 raft_consensus.cc:348] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.630019 25957 raft_consensus.cc:374] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:33:26.630057 25957 raft_consensus.cc:729] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e4a45a478d49dcb79a379773bdcb0b, State: Initialized, Role: FOLLOWER
I20260809 19:33:26.630028 25959 consensus_queue.cc:260] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.630156 25957 consensus_queue.cc:260] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.630308 25959 ts_tablet_manager.cc:1428] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:33:26.630347 25739 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
I20260809 19:33:26.630409 25935 heartbeater.cc:503] Master 127.31.57.254:33311 was elected leader, sending a full tablet report...
I20260809 19:33:26.630409 25957 ts_tablet_manager.cc:1428] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 19:33:26.694120 25740 tablet.cc:2369] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:33:26.795989 25524 tablet.cc:2369] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:33:26.818035 25965 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:26.818162 25965 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.818575 25965 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185), b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:26.822435 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.818838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:59071 (local address 127.31.57.193:33503)
0809 19:33:26.818883 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.818885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.819019 (+ 134us) server_negotiation.cc:407] Connection header received
0809 19:33:26.819109 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.819110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.819129 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.819203 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:26.819350 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.820025 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.821519 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.822102 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.822165 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.822231 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.822337 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.822373 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:33:26.822435 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.818857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54133 (local address 127.31.57.194:39185)
0809 19:33:26.818904 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.818906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.818915 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:33:26.818950 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.818950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.818958 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.819007 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:33:26.819218 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.820001 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.821517 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.822084 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.822165 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.822173 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.822335 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.822372 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:33:26.822444 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.818731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39185 (local address 127.31.57.195:54133)
0809 19:33:26.818789 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.818800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.818848 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.819053 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.819054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.819056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:26.819136 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.819138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.820093 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.820094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.821350 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.821354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.822171 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.822172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.822258 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.822259 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.822316 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.822354 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":109}
I20260809 19:33:26.822450 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.818786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.31.57.195:59071)
0809 19:33:26.818954 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.818962 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:26.819031 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:26.819214 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:26.819215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:26.819216 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:33:26.819277 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.819279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.820095 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.820095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.821350 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:26.821354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.822172 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.822172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:26.822253 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.822257 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:33:26.822312 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:33:26.822354 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":16,"thread_start_us":91,"threads_started":1}
I20260809 19:33:26.822825 25444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e073e977005455f9fe189d287c786f3" candidate_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36edeb0c7c245d9b784a377c65a9d58" is_pre_election: true
I20260809 19:33:26.823019 25444 raft_consensus.cc:2454] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e4a45a478d49dcb79a379773bdcb0b in term 0.
I20260809 19:33:26.822997 25667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e073e977005455f9fe189d287c786f3" candidate_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" is_pre_election: true
I20260809 19:33:26.823117 25667 raft_consensus.cc:2454] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e4a45a478d49dcb79a379773bdcb0b in term 0.
I20260809 19:33:26.823349 25764 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58, b6e4a45a478d49dcb79a379773bdcb0b; no voters:
I20260809 19:33:26.823504 25965 raft_consensus.cc:2781] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:33:26.823594 25965 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:33:26.823629 25965 raft_consensus.cc:3037] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:26.824288 25965 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.824470 25965 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [CANDIDATE]: Term 1 election: Requested vote from peers ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185), b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503)
I20260809 19:33:26.824620 25667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e073e977005455f9fe189d287c786f3" candidate_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
I20260809 19:33:26.824623 25444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e073e977005455f9fe189d287c786f3" candidate_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36edeb0c7c245d9b784a377c65a9d58"
I20260809 19:33:26.824716 25444 raft_consensus.cc:3037] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:26.824720 25667 raft_consensus.cc:3037] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:33:26.825412 25444 raft_consensus.cc:2454] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e4a45a478d49dcb79a379773bdcb0b in term 1.
I20260809 19:33:26.825417 25667 raft_consensus.cc:2454] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e4a45a478d49dcb79a379773bdcb0b in term 1.
I20260809 19:33:26.825572 25764 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58, b6e4a45a478d49dcb79a379773bdcb0b; no voters:
I20260809 19:33:26.825675 25965 raft_consensus.cc:2781] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:33:26.825796 25965 raft_consensus.cc:686] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 1 LEADER]: Becoming Leader. State: Replica: b6e4a45a478d49dcb79a379773bdcb0b, State: Running, Role: LEADER
I20260809 19:33:26.825894 25965 consensus_queue.cc:237] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:26.826669 25230 catalog_manager.cc:5495] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b reported cstate change: term changed from 0 to 1, leader changed from <none> to b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195). New cstate: current_term: 1 leader_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } health_report { overall_health: UNKNOWN } } }
W20260809 19:33:26.841823 25946 tablet.cc:2369] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:33:26.847815 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.845423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.31.57.195:32799)
0809 19:33:26.845480 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:26.845483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:26.845492 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:33:26.845562 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:26.845563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:26.845578 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:26.845637 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:33:26.845744 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.846230 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.846890 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:26.847083 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:26.847153 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:26.847335 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:33:26.847553 (+ 218us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:33:26.847637 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:26.847696 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:33:26.847716 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:33:26.849433 25667 raft_consensus.cc:1261] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Refusing update from remote peer b6e4a45a478d49dcb79a379773bdcb0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:33:26.849462 25444 raft_consensus.cc:1261] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Refusing update from remote peer b6e4a45a478d49dcb79a379773bdcb0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:33:26.849798 25969 consensus_queue.cc:1035] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:33:26.849905 25965 consensus_queue.cc:1035] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:33:26.852253 25977 mvcc.cc:205] Tried to move back new op lower bound from 7316701212052742144 to 7316701211959726080. Current Snapshot: MvccSnapshot[applied={T|T < 7316701212052742144}]
I20260809 19:33:26.854247 25230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:33:26.854326 25230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:33:26.855017 25230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Sending DeleteTablet for 3 replicas of tablet 1e073e977005455f9fe189d287c786f3
I20260809 19:33:26.855338 25417 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e073e977005455f9fe189d287c786f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:26 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50692
I20260809 19:33:26.855336 25630 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e073e977005455f9fe189d287c786f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:26 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:44344
I20260809 19:33:26.855413 25833 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e073e977005455f9fe189d287c786f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:33:26 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:60658
I20260809 19:33:26.855542 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 25291
I20260809 19:33:26.855608 25992 tablet_replica.cc:330] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: stopping tablet replica
I20260809 19:33:26.855697 25992 raft_consensus.cc:2229] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:33:26.855780 25992 raft_consensus.cc:2258] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:33:26.855901 25993 tablet_replica.cc:330] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: stopping tablet replica
I20260809 19:33:26.855976 25993 raft_consensus.cc:2229] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:33:26.856047 25993 raft_consensus.cc:2258] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:33:26.861477 25992 ts_tablet_manager.cc:1905] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:33:26.863198 25992 ts_tablet_manager.cc:1918] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:33:26.863327 25992 log.cc:1196] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/1e073e977005455f9fe189d287c786f3
I20260809 19:33:26.863723 25992 ts_tablet_manager.cc:1939] T 1e073e977005455f9fe189d287c786f3 P ed3d2d60e71343fcaee9e5a7a65d5d31: Deleting consensus metadata
I20260809 19:33:26.864171 25207 catalog_manager.cc:4841] TS ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): tablet 1e073e977005455f9fe189d287c786f3 (table test-table [id=70f145bb3bdf4f558090fda740a1b91a]) successfully deleted
I20260809 19:33:26.867286 25993 ts_tablet_manager.cc:1905] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 19:33:26.868952 25209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:33503 (error 108)
I20260809 19:33:26.869374 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 25531
I20260809 19:33:26.869441 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:26.869306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33503 (local address 127.0.0.1:50700)
0809 19:33:26.869361 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:26.869404 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:33:26.887046 25993 ts_tablet_manager.cc:1918] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:33:26.887135 25993 log.cc:1196] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/1e073e977005455f9fe189d287c786f3
I20260809 19:33:26.887493 25993 ts_tablet_manager.cc:1939] T 1e073e977005455f9fe189d287c786f3 P b6e4a45a478d49dcb79a379773bdcb0b: Deleting consensus metadata
W20260809 19:33:26.888909 25209 catalog_manager.cc:4584] TS b36edeb0c7c245d9b784a377c65a9d58 (127.31.57.193:33503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e073e977005455f9fe189d287c786f3: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33503: connect: Connection refused (error 111)
I20260809 19:33:26.889394 25209 catalog_manager.cc:4841] TS b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): tablet 1e073e977005455f9fe189d287c786f3 (table test-table [id=70f145bb3bdf4f558090fda740a1b91a]) successfully deleted
I20260809 19:33:26.893603 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 25743
I20260809 19:33:26.902182 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 25185
I20260809 19:33:46.911654 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--webserver_interface=127.31.57.254
--webserver_port=46045
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:46.990166 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:46.990343 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:46.990365 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:46.991441 25996 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:33:46.991482 25996 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:33:46.991493 25996 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:33:46.991504 25996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:33:46.991516 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:46.991528 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:46.991539 25996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:33:46.991552 25996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:33:46.992812 25996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=46045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:46.992964 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:46.993114 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:46.997952 26006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:46.997958 26005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:46.998076 25996 server_base.cc:1034] running on GCE node
W20260809 19:33:46.997956 26008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:46.998468 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:46.998777 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:46.999919 25996 hybrid_clock.cc:648] HybridClock initialized: now 1786304026999905 us; error 41 us; skew 500 ppm
Aug 09 19:33:47 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:47.005470 25996 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:33:47.005857 25996 webserver.cc:460] Webserver started at http://127.31.57.254:46045/ using document root <none> and password file <none>
I20260809 19:33:47.006038 25996 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:47.006076 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:47.007221 25996 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:47.007956 26017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.008185 25996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:47.008276 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "05cb3d76d62742249b9bee44f62145ae"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "bb4e9c47e7c03486e74878f46b609dd5"
server_key_iv: "b338847b990f5e938bfea7cb7df1c19c"
server_key_version: "encryptionkey@0"
I20260809 19:33:47.008589 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:47.030372 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:47.032168 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:47.032333 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:47.036219 25996 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:33311
I20260809 19:33:47.036248 26098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:33311 every 8 connection(s)
I20260809 19:33:47.036576 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:33:47.037549 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 25996
I20260809 19:33:47.037850 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:33503
--local_ip_for_outbound_sockets=127.31.57.193
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=39013
--webserver_interface=127.31.57.193
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:33:47.038331 26099 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:33:47.039404 26099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap starting.
I20260809 19:33:47.048604 26099 log.cc:824] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:47.050611 26099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:33:47.050912 26099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Bootstrap complete.
I20260809 19:33:47.053134 26099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:47.053341 26099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Initialized, Role: FOLLOWER
I20260809 19:33:47.053542 26099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:47.053599 26099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:33:47.053660 26099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:33:47.053700 26099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:33:47.054576 26099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:47.054656 26099 leader_election.cc:302] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05cb3d76d62742249b9bee44f62145ae; no voters:
I20260809 19:33:47.054855 26099 leader_election.cc:288] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:33:47.054886 26107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:33:47.054991 26107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [term 3 LEADER]: Becoming Leader. State: Replica: 05cb3d76d62742249b9bee44f62145ae, State: Running, Role: LEADER
I20260809 19:33:47.055099 26099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:33:47.055122 26107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } }
I20260809 19:33:47.055516 26109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:47.055619 26109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:47.055896 26107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05cb3d76d62742249b9bee44f62145ae. Latest consensus state: current_term: 3 leader_uuid: "05cb3d76d62742249b9bee44f62145ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05cb3d76d62742249b9bee44f62145ae" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 33311 } } }
I20260809 19:33:47.055959 26107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:33:47.056027 26122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:33:47.056833 26122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:33:47.056949 26122 catalog_manager.cc:1259] Loaded cluster ID: 53cf4f86e8604c14942c7868ceb8bb2a
I20260809 19:33:47.057185 26122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:33:47.057909 26122 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:33:47.058087 26122 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Loaded TSK: 0
I20260809 19:33:47.058123 26122 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05cb3d76d62742249b9bee44f62145ae: Loaded TSK: 1
F20260809 19:33:47.058318 26122 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fad559749ad google::LogMessage::Fail() at ??:0
@ 0x7fad5597788a google::LogMessage::SendToLog() at ??:0
@ 0x7fad559744de google::LogMessage::Flush() at ??:0
@ 0x7fad55975a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fad580e1104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fad580ded41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fad5d8ce1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fad5d8ce48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fad5d8ce594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fad5d894c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fad5d89ead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fad578420d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fad57837954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fad5869b6db start_thread at ??:0
@ 0x7fad5447261f clone at ??:0
W20260809 19:33:47.135526 26103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:47.135707 26103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:47.135730 26103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:47.136817 26103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:47.136853 26103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:47.136878 26103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:33:47.138338 26103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:33503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=39013
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:47.138584 26103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:47.138823 26103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:47.143857 26167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:47.143965 26103 server_base.cc:1034] running on GCE node
W20260809 19:33:47.143975 26169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:47.144114 26174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:47.144320 26103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:47.144622 26103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:47.145793 26103 hybrid_clock.cc:648] HybridClock initialized: now 1786304027145776 us; error 31 us; skew 500 ppm
Aug 09 19:33:47 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:47.148181 26103 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:33:47.148646 26103 webserver.cc:460] Webserver started at http://127.31.57.193:39013/ using document root <none> and password file <none>
I20260809 19:33:47.148891 26103 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:47.148947 26103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:47.150259 26103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:33:47.151034 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.151245 26103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:33:47.151353 26103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "b36edeb0c7c245d9b784a377c65a9d58"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "1a062e0cb76c625abc3c2c919b7ad840"
server_key_iv: "95e6b4c15e27232ccb596f042b73b7c5"
server_key_version: "encryptionkey@0"
I20260809 19:33:47.151703 26103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:47.162659 26103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:47.164729 26103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:47.164892 26103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:47.165102 26103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:47.165508 26199 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:33:47.167233 26103 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:33:47.167289 26103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:33:47.167325 26103 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:33:47.167920 26103 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:33:47.167976 26103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:47.168056 26199 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap starting.
I20260809 19:33:47.174857 26103 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:33503
I20260809 19:33:47.174993 26374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:33503 every 8 connection(s)
I20260809 19:33:47.175266 26103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:33:47.179185 26199 log.cc:824] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Log is configured to *not* fsync() on all Append() calls
I20260809 19:33:47.181669 26199 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:33:47.181969 26199 tablet_bootstrap.cc:492] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Bootstrap complete.
I20260809 19:33:47.183231 26199 ts_tablet_manager.cc:1397] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260809 19:33:47.183739 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 26103
I20260809 19:33:47.184031 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:39185
--local_ip_for_outbound_sockets=127.31.57.194
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=34371
--webserver_interface=127.31.57.194
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:33:47.184839 26199 raft_consensus.cc:348] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:47.185175 26199 raft_consensus.cc:729] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36edeb0c7c245d9b784a377c65a9d58, State: Initialized, Role: FOLLOWER
I20260809 19:33:47.185338 26199 consensus_queue.cc:260] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:47.185629 26199 ts_tablet_manager.cc:1428] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
W20260809 19:33:47.245348 26379 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:33:47.175923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:51417)
0809 19:33:47.176105 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.176143 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:47.176537 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:47.245106 (+ 68569us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
W20260809 19:33:47.245671 26375 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33311 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:33311: Client connection negotiation failed: client connection to 127.31.57.254:33311: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:33:47.246088 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.245951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:40895)
0809 19:33:47.246014 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.246061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:33:47.246474 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.246372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:54935)
0809 19:33:47.246422 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.246450 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:33:47.246608 26375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:33:47.263909 26383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:47.264076 26383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:47.264098 26383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:47.265270 26383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:47.265312 26383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:47.265338 26383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:33:47.266767 26383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=34371
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:47.267019 26383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:47.267251 26383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:47.271914 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:47.271889 26396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:47.271893 26398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:47.272149 26383 server_base.cc:1034] running on GCE node
I20260809 19:33:47.272369 26383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:47.272594 26383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:47.273775 26383 hybrid_clock.cc:648] HybridClock initialized: now 1786304027273770 us; error 38 us; skew 500 ppm
Aug 09 19:33:47 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:47.276046 26383 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:33:47.276495 26383 webserver.cc:460] Webserver started at http://127.31.57.194:34371/ using document root <none> and password file <none>
I20260809 19:33:47.276732 26383 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:47.276803 26383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:47.277968 26383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:33:47.278537 26406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.278702 26383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:33:47.278798 26383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "68a490665bd7875af942ebc20c4e42bb"
server_key_iv: "9291118ba6b1dea93c62069a13de0061"
server_key_version: "encryptionkey@0"
I20260809 19:33:47.279088 26383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:47.297629 26383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:47.299496 26383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:47.299654 26383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:47.299912 26383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:47.300230 26383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:47.300292 26383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.300333 26383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:47.300359 26383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.306551 26383 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:39185
I20260809 19:33:47.306617 26582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:39185 every 8 connection(s)
I20260809 19:33:47.306931 26383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:33:47.307852 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.307529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:42721)
0809 19:33:47.307654 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.307722 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
W20260809 19:33:47.308264 26583 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33311 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:33311: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
I20260809 19:33:47.308522 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.308440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:45843)
0809 19:33:47.308469 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.308493 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 19:33:47.308799 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.308744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:46947)
0809 19:33:47.308767 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.308785 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 19:33:47.308921 26583 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:33:47.309809 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 26383
I20260809 19:33:47.310223 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:32799
--local_ip_for_outbound_sockets=127.31.57.195
--tserver_master_addrs=127.31.57.254:33311
--webserver_port=36465
--webserver_interface=127.31.57.195
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:33:47.391019 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:33:47.391252 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:33:47.391291 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:33:47.392426 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:33:47.392501 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:33:47.392545 26587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:33:47.393946 26587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:32799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=36465
--tserver_master_addrs=127.31.57.254:33311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:33:47.394200 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:33:47.394480 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:33:47.399410 26599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:33:47.399480 26602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:47.399596 26587 server_base.cc:1034] running on GCE node
W20260809 19:33:47.399411 26600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:33:47.399859 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:33:47.400103 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:33:47.401244 26587 hybrid_clock.cc:648] HybridClock initialized: now 1786304027401228 us; error 27 us; skew 500 ppm
Aug 09 19:33:47 dist-test-slave-x4qh krb5kdc[24371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:33:47.403721 26587 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:33:47.404175 26587 webserver.cc:460] Webserver started at http://127.31.57.195:36465/ using document root <none> and password file <none>
I20260809 19:33:47.404443 26587 fs_manager.cc:359] Metadata directory not provided
I20260809 19:33:47.404524 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:33:47.405923 26587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:33:47.406540 26617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.406721 26587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:33:47.406836 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "b6e4a45a478d49dcb79a379773bdcb0b"
format_stamp: "Formatted at 2026-08-09 19:33:05 on dist-test-slave-x4qh"
server_key: "6643af3689da3783cfe497b0889c03cf"
server_key_iv: "13afdd61a08a8d40ca1b69bef50d24b8"
server_key_version: "encryptionkey@0"
I20260809 19:33:47.407140 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:33:47.418465 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:33:47.420275 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:33:47.420454 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:33:47.420683 26587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:33:47.420993 26587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:33:47.421056 26587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.421095 26587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:33:47.421123 26587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:33:47.427575 26587 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:32799
I20260809 19:33:47.427629 26806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:32799 every 8 connection(s)
I20260809 19:33:47.428005 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:33:47.429011 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.428676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:39497)
0809 19:33:47.428825 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.428886 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
W20260809 19:33:47.429425 26807 heartbeater.cc:644] Failed to heartbeat to 127.31.57.254:33311 (0 consecutive failures): Network error: Failed to ping master at 127.31.57.254:33311: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
I20260809 19:33:47.429651 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.429590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:37571)
0809 19:33:47.429615 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.429637 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 19:33:47.429898 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:47.429852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:50389)
0809 19:33:47.429870 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:47.429887 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260809 19:33:47.430006 26807 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:33:47.436101 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 26587
I20260809 19:33:48.247198 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.246955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:51507)
0809 19:33:48.247099 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:48.247140 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260809 19:33:48.309526 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.309264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:60371)
0809 19:33:48.309446 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:48.309478 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260809 19:33:48.373433 26823 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:48.373562 26823 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:48.374001 26823 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
Aug 09 19:33:48 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.195@KRBTEST.COM
Aug 09 19:33:48 dist-test-slave-x4qh krb5kdc[24371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304027, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.194@KRBTEST.COM
I20260809 19:33:48.384124 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.374339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:37071 (local address 127.31.57.195:32799)
0809 19:33:48.374477 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:48.374489 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:48.374511 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:33:48.374916 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:48.374918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:48.374967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:48.375073 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:48.375934 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.376735 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.377643 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.378004 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.378106 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:48.380454 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:48.380457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:48.380457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:48.380466 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:48.382381 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:48.382764 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:48.382766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:48.382767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:48.382896 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:48.383197 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:48.383198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:48.383198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:48.383506 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:48.383605 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:48.383901 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 19:33:48.384023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260809 19:33:48.384133 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.374137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:32799 (local address 127.31.57.193:37071)
0809 19:33:48.374215 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:48.374225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:48.374311 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:48.375152 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:48.375155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:48.375612 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:48.375793 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:48.375800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.376862 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.376863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:48.377545 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:48.377548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.378099 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.378100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:48.378203 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:48.378740 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:48.378748 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:48.380334 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:48.382456 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:48.382479 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:48.382481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:48.382698 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:48.382993 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:48.382994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:48.382995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:48.383104 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:48.383586 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:48.383594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:48.383726 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:33:48.383847 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 19:33:48.384007 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":58}
I20260809 19:33:48.384227 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.374186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:39185 (local address 127.31.57.193:49911)
0809 19:33:48.374343 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:48.374353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:33:48.374407 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:33:48.375158 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:33:48.375160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:33:48.375613 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:33:48.375795 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:48.375800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.376828 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.376829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:48.377545 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:33:48.377547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.378132 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.378132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:33:48.378204 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:48.378780 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:33:48.378793 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:33:48.380761 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:33:48.382906 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:48.382907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:48.382907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:48.383070 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:48.383349 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:33:48.383350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:33:48.383350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:33:48.383448 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:33:48.383845 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:33:48.383846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:33:48.383892 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 19:33:48.383981 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:33:48.384082 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260809 19:33:48.384325 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.374347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:49911 (local address 127.31.57.194:39185)
0809 19:33:48.374508 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 19:33:48.374524 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 19:33:48.374546 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:33:48.374949 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:33:48.374951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:33:48.374983 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:33:48.375135 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:33:48.375926 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.376714 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.377646 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:33:48.378017 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:33:48.378155 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:33:48.380871 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:33:48.380874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:33:48.380874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:33:48.380883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:33:48.382766 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:48.383162 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:48.383165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:48.383165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:48.383262 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:33:48.383522 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:33:48.383524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:33:48.383524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:33:48.383757 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:33:48.383860 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 19:33:48.384061 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 19:33:48.384194 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
W20260809 19:33:48.385177 26187 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
W20260809 19:33:48.385371 26189 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
I20260809 19:33:48.385442 26189 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58; no voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31
I20260809 19:33:48.385574 26823 raft_consensus.cc:2726] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:48.430469 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:48.430299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:39193)
0809 19:33:48.430400 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:48.430438 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260809 19:33:49.247776 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:49.247568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:43241)
0809 19:33:49.247689 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:49.247722 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260809 19:33:49.310112 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:49.309923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:49253)
0809 19:33:49.310045 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:49.310079 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260809 19:33:49.431116 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:49.430900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:39959)
0809 19:33:49.431022 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:49.431072 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260809 19:33:50.113040 26836 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:50.113135 26836 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:50.113340 26836 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
W20260809 19:33:50.113731 26187 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
W20260809 19:33:50.113823 26189 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
I20260809 19:33:50.113876 26189 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58; no voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31
I20260809 19:33:50.113962 26836 raft_consensus.cc:2726] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:50.248358 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:50.248140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:35093)
0809 19:33:50.248277 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:50.248313 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260809 19:33:50.310673 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:50.310496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:39241)
0809 19:33:50.310606 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:50.310637 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260809 19:33:50.431672 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:50.431462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:56297)
0809 19:33:50.431607 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:50.431639 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260809 19:33:51.248960 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:51.248761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:59529)
0809 19:33:51.248880 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:51.248914 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260809 19:33:51.311240 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:51.311063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:60441)
0809 19:33:51.311171 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:51.311203 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260809 19:33:51.432312 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:51.432121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:56745)
0809 19:33:51.432245 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:51.432278 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260809 19:33:52.249568 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:52.249368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:43295)
0809 19:33:52.249489 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:52.249523 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260809 19:33:52.311821 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:52.311647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:52149)
0809 19:33:52.311756 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:52.311787 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260809 19:33:52.432865 26855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:52.432678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:48263)
0809 19:33:52.432796 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:52.432831 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260809 19:33:52.724414 26858 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:52.724506 26858 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:52.724692 26858 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
W20260809 19:33:52.725044 26187 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
W20260809 19:33:52.725129 26189 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
I20260809 19:33:52.725214 26189 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58; no voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31
I20260809 19:33:52.725306 26858 raft_consensus.cc:2726] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:53.250227 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:53.250002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:41997)
0809 19:33:53.250132 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:53.250171 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260809 19:33:53.312371 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:53.312194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:44567)
0809 19:33:53.312301 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:53.312334 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260809 19:33:53.433446 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:53.433271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:58921)
0809 19:33:53.433381 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:53.433411 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260809 19:33:54.250979 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:54.250747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:45837)
0809 19:33:54.250898 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:54.250934 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260809 19:33:54.312999 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:54.312814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:46305)
0809 19:33:54.312930 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:54.312963 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260809 19:33:54.434037 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:54.433845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:49419)
0809 19:33:54.433972 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:54.434002 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260809 19:33:55.046396 26867 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:55.046523 26867 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:55.046725 26867 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
W20260809 19:33:55.047063 26189 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
W20260809 19:33:55.047127 26187 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
I20260809 19:33:55.047214 26187 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58; no voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31
I20260809 19:33:55.047302 26867 raft_consensus.cc:2726] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:55.251811 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:55.251575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:34209)
0809 19:33:55.251731 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:55.251766 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260809 19:33:55.313580 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:55.313365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:55973)
0809 19:33:55.313492 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:55.313526 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260809 19:33:55.434648 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:55.434364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:48845)
0809 19:33:55.434567 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:55.434601 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
I20260809 19:33:56.252496 26873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:56.252290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:47139)
0809 19:33:56.252418 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:56.252452 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260809 19:33:56.314136 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:56.313946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:39481)
0809 19:33:56.314071 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:56.314101 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260809 19:33:56.435257 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:56.435067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:40147)
0809 19:33:56.435186 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:56.435219 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260809 19:33:57.253192 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:57.252974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:50187)
0809 19:33:57.253119 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:57.253150 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260809 19:33:57.314757 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:57.314565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:45847)
0809 19:33:57.314690 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:57.314719 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260809 19:33:57.435865 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:57.435676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:57157)
0809 19:33:57.435790 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:57.435825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260809 19:33:58.253746 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:58.253561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:35345)
0809 19:33:58.253675 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:58.253702 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260809 19:33:58.315434 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:58.315178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:39417)
0809 19:33:58.315307 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:58.315339 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260809 19:33:58.436489 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:58.436285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:48903)
0809 19:33:58.436414 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:58.436448 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260809 19:33:59.112596 26883 raft_consensus.cc:482] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:33:59.112685 26883 raft_consensus.cc:504] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e4a45a478d49dcb79a379773bdcb0b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 32799 } } peers { permanent_uuid: "ed3d2d60e71343fcaee9e5a7a65d5d31" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 39185 } } peers { permanent_uuid: "b36edeb0c7c245d9b784a377c65a9d58" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33503 } }
I20260809 19:33:59.112882 26883 leader_election.cc:288] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799), ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185)
W20260809 19:33:59.113251 26187 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed3d2d60e71343fcaee9e5a7a65d5d31 (127.31.57.194:39185): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
W20260809 19:33:59.113350 26189 leader_election.cc:341] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b6e4a45a478d49dcb79a379773bdcb0b (127.31.57.195:32799): Not found: Tablet not found: 1e073e977005455f9fe189d287c786f3
I20260809 19:33:59.113384 26189 leader_election.cc:302] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b36edeb0c7c245d9b784a377c65a9d58; no voters: b6e4a45a478d49dcb79a379773bdcb0b, ed3d2d60e71343fcaee9e5a7a65d5d31
I20260809 19:33:59.113466 26883 raft_consensus.cc:2726] T 1e073e977005455f9fe189d287c786f3 P b36edeb0c7c245d9b784a377c65a9d58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:33:59.254467 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:59.254262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:54859)
0809 19:33:59.254387 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:59.254422 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260809 19:33:59.316105 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:59.315879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:42129)
0809 19:33:59.315989 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:59.316029 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":61,"threads_started":1}
I20260809 19:33:59.437134 26887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:33:59.436904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:34001)
0809 19:33:59.437065 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:33:59.437095 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":57,"threads_started":1}
I20260809 19:34:00.255090 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:00.254881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:52927)
0809 19:34:00.254998 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:00.255030 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260809 19:34:00.316702 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:00.316524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:54033)
0809 19:34:00.316634 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:00.316665 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260809 19:34:00.437690 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:00.437513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:40813)
0809 19:34:00.437624 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:00.437655 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260809 19:34:01.255729 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:01.255532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:60485)
0809 19:34:01.255654 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:01.255685 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260809 19:34:01.317364 26893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:01.317166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:34065)
0809 19:34:01.317295 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:01.317328 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260809 19:34:01.438264 26894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:01.438081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:55339)
0809 19:34:01.438199 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:01.438231 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260809 19:34:02.256335 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:02.256140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.193:46735)
0809 19:34:02.256251 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:02.256291 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 19:34:02.318143 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:02.317964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.194:42285)
0809 19:34:02.318077 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:02.318108 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260809 19:34:02.438835 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:02.438665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:33311 (local address 127.31.57.195:36025)
0809 19:34:02.438768 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:02.438803 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.254:33311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260809 19:34:02.446405 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 26103
I20260809 19:34:02.460830 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 26383
I20260809 19:34:02.468916 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 26587
2026-08-09T19:34:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58584 ms)
[----------] 30 tests from SecurityITest (231131 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:02 dist-test-slave-x4qh krb5kdc[26907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:02 dist-test-slave-x4qh krb5kdc[26907](info): set up 2 sockets
Aug 09 19:34:02 dist-test-slave-x4qh krb5kdc[26907](info): commencing operation
krb5kdc: starting...
W20260809 19:34:03.643857 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:03 dist-test-slave-x4qh krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:03.761407 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34343
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:03.839334 26927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:03.839605 26927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:03.839639 26927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:03.840718 26927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:03.840796 26927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:03.840823 26927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:03.840852 26927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:03.840880 26927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:03.842290 26927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:03.842555 26927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:03.842772 26927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:03.847651 26935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:03.847667 26938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:03.847675 26927 server_base.cc:1034] running on GCE node
W20260809 19:34:03.847702 26933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:03.848026 26927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:03.848318 26927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:03.849478 26927 hybrid_clock.cc:648] HybridClock initialized: now 1786304043849466 us; error 43 us; skew 500 ppm
Aug 09 19:34:03 dist-test-slave-x4qh krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:03.851908 26927 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:03.852327 26927 webserver.cc:460] Webserver started at http://127.31.57.254:42503/ using document root <none> and password file <none>
I20260809 19:34:03.852572 26927 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:03.852643 26927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:03.852811 26927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:03.853865 26927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "96764a0933a74698bead3de46e1d7680"
format_stamp: "Formatted at 2026-08-09 19:34:03 on dist-test-slave-x4qh"
server_key: "dc9c9ce9775342ef9a04bda2369ab8a7"
server_key_iv: "f438923529c9aef2f7943532da86a228"
server_key_version: "encryptionkey@0"
I20260809 19:34:03.854231 26927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "96764a0933a74698bead3de46e1d7680"
format_stamp: "Formatted at 2026-08-09 19:34:03 on dist-test-slave-x4qh"
server_key: "dc9c9ce9775342ef9a04bda2369ab8a7"
server_key_iv: "f438923529c9aef2f7943532da86a228"
server_key_version: "encryptionkey@0"
I20260809 19:34:03.855592 26927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:03.856308 26947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:03.856495 26927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:03.856592 26927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "96764a0933a74698bead3de46e1d7680"
format_stamp: "Formatted at 2026-08-09 19:34:03 on dist-test-slave-x4qh"
server_key: "dc9c9ce9775342ef9a04bda2369ab8a7"
server_key_iv: "f438923529c9aef2f7943532da86a228"
server_key_version: "encryptionkey@0"
I20260809 19:34:03.856678 26927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:03.891263 26927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:03.893172 26927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:03.893335 26927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:03.897327 26927 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34343
I20260809 19:34:03.897372 27040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34343 every 8 connection(s)
I20260809 19:34:03.897693 26927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:03.898518 27041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:03.900687 27041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: Bootstrap starting.
I20260809 19:34:03.901292 27041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:03.901584 27041 log.cc:824] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:03.902284 27041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: No bootstrap required, opened a new log
I20260809 19:34:03.903772 27041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } }
I20260809 19:34:03.903925 27041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:03.903947 27041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96764a0933a74698bead3de46e1d7680, State: Initialized, Role: FOLLOWER
I20260809 19:34:03.904121 27041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } }
I20260809 19:34:03.904202 27041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:03.904250 27041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:03.904295 27041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:03.904820 27041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } }
I20260809 19:34:03.904922 27041 leader_election.cc:302] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96764a0933a74698bead3de46e1d7680; no voters:
I20260809 19:34:03.905113 27041 leader_election.cc:288] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:03.905151 27046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:03.905329 27046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [term 1 LEADER]: Becoming Leader. State: Replica: 96764a0933a74698bead3de46e1d7680, State: Running, Role: LEADER
I20260809 19:34:03.905364 27041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:03.905441 27046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } }
I20260809 19:34:03.905851 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96764a0933a74698bead3de46e1d7680. Latest consensus state: current_term: 1 leader_uuid: "96764a0933a74698bead3de46e1d7680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } } }
I20260809 19:34:03.905970 27050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96764a0933a74698bead3de46e1d7680" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96764a0933a74698bead3de46e1d7680" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34343 } } }
I20260809 19:34:03.906054 27050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:03.906268 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 26927
I20260809 19:34:03.906385 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:03.906010 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:03.906589 31975 external_mini_cluster.cc:1428] Setting key f6b6b6c35d7968c5b02e97881cb0928d
W20260809 19:34:03.907891 27073 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:03.907950 27073 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:03.908004 27062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:03.908180 27062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:03.909587 27062 catalog_manager.cc:1347] Generated new cluster ID: bf4c83319bd6413fb09c3455b07a04da
I20260809 19:34:03.909659 27062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:03 dist-test-slave-x4qh krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:03.916320 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:03.907214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.31.57.254:34343)
0809 19:34:03.907459 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:03.907470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:03.907503 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:34:03.907621 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:03.907629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:03.907675 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:03.907805 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:03.908537 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:03.909836 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:03.910551 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:03.910936 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:03.911048 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:03.913261 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:03.913264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:03.913264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:03.913287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:03.914839 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:03.915201 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:03.915204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:03.915205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:03.915309 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:03.915561 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:03.915562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:03.915562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:03.915771 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:03.915930 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:03.916003 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:34:03.916130 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260809 19:34:03.928786 27062 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:03.929451 27062 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:03.935020 27062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: Generated new TSK 0
I20260809 19:34:03.935179 27062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:34:03.963784 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:04.055994 27083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.056164 27083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.056185 27083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.057258 27083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.057297 27083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.057322 27083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:04.058749 27083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.058980 27083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.059231 27083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.064219 27094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.064225 27093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.064241 27083 server_base.cc:1034] running on GCE node
W20260809 19:34:04.064231 27097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.064663 27083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.064857 27083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.065982 27083 hybrid_clock.cc:648] HybridClock initialized: now 1786304044065973 us; error 27 us; skew 500 ppm
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:04.068295 27083 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:34:04.068686 27083 webserver.cc:460] Webserver started at http://127.31.57.193:40727/ using document root <none> and password file <none>
I20260809 19:34:04.068953 27083 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.068997 27083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.069164 27083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.070111 27083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "e7287b16bb74470d82d3aa442ef4cc31"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "f82a247d947bb6f5fbdee7cbe4a21fc1"
server_key_iv: "5eb7aad51cc396a6727b780edba77ed0"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.070473 27083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "e7287b16bb74470d82d3aa442ef4cc31"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "f82a247d947bb6f5fbdee7cbe4a21fc1"
server_key_iv: "5eb7aad51cc396a6727b780edba77ed0"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.071770 27083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:34:04.072448 27111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.072611 27083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:04.072700 27083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "e7287b16bb74470d82d3aa442ef4cc31"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "f82a247d947bb6f5fbdee7cbe4a21fc1"
server_key_iv: "5eb7aad51cc396a6727b780edba77ed0"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.072788 27083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:04.082877 27083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:04.084503 27083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:04.084650 27083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:04.084849 27083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:04.085155 27083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:04.085187 27083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.085235 27083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:04.085261 27083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.090758 27083 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:44523
I20260809 19:34:04.090843 27305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:44523 every 8 connection(s)
I20260809 19:34:04.091118 27083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:04.098780 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 27083
I20260809 19:34:04.098892 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:04.099104 31975 external_mini_cluster.cc:1428] Setting key d2000e57be519cdfd1f4cde1ce8835eb
I20260809 19:34:04.099848 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.091816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:51001 (local address 127.31.57.254:34343)
0809 19:34:04.091885 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.091887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.092085 (+ 198us) server_negotiation.cc:407] Connection header received
0809 19:34:04.092598 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.092599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.092616 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.092672 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.093557 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.094046 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.094954 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.095156 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.095238 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.097261 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.097262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.097262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:04.097268 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.098395 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.098744 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.098745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.098745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.098808 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.099112 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.099113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.099114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.099263 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.099335 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.099641 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.099754 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 19:34:04.099960 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.091804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34343 (local address 127.31.57.193:51001)
0809 19:34:04.091972 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.092003 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.092487 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.092742 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.092745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.093271 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:04.093450 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.093455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.094142 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.094143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.094860 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.094862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.095252 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.095253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.095393 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.095849 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:04.095857 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:04.097180 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:04.098465 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.098471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.098473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.098666 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.098887 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.098888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.098889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.099049 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.099345 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:04.099352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:04.099521 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.099600 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.099749 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260809 19:34:04.100440 27307 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34343
I20260809 19:34:04.100533 27307 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:04.100703 27307 heartbeater.cc:511] Master 127.31.57.254:34343 requested a full tablet report, sending...
I20260809 19:34:04.101162 26986 ts_manager.cc:194] Registered new tserver with Master: e7287b16bb74470d82d3aa442ef4cc31 (127.31.57.193:44523)
I20260809 19:34:04.101750 26986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:51001
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:34:04.123664 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:04.201376 27315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.201558 27315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.201579 27315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.202649 27315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.202705 27315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.202731 27315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:04.204151 27315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.204411 27315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.204638 27315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.209367 27327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.209381 27323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.209452 27315 server_base.cc:1034] running on GCE node
W20260809 19:34:04.209380 27324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.209736 27315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.209968 27315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.211126 27315 hybrid_clock.cc:648] HybridClock initialized: now 1786304044211118 us; error 39 us; skew 500 ppm
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:04.213472 27315 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:34:04.213903 27315 webserver.cc:460] Webserver started at http://127.31.57.194:38391/ using document root <none> and password file <none>
I20260809 19:34:04.214141 27315 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.214205 27315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.214332 27315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.215214 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "67d5228a75914e49ae78a486ba67ef23"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5c0740ce14f64194b9c15dc8e889ad"
server_key_iv: "76ae227251abee9c0420872eb7862ad1"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.215574 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "67d5228a75914e49ae78a486ba67ef23"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5c0740ce14f64194b9c15dc8e889ad"
server_key_iv: "76ae227251abee9c0420872eb7862ad1"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.216809 27315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:34:04.217489 27340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.217662 27315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:04.217753 27315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "67d5228a75914e49ae78a486ba67ef23"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5c0740ce14f64194b9c15dc8e889ad"
server_key_iv: "76ae227251abee9c0420872eb7862ad1"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.217835 27315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:04.232870 27315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:04.234592 27315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:04.234738 27315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:04.234942 27315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:04.235224 27315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:04.235283 27315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.235317 27315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:04.235342 27315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.240649 27315 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:44983
I20260809 19:34:04.240680 27541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:44983 every 8 connection(s)
I20260809 19:34:04.240952 27315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:04.247879 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 27315
I20260809 19:34:04.247998 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:04.248184 31975 external_mini_cluster.cc:1428] Setting key 71762d6ae43edc6bbe93eb77e2c2a387
I20260809 19:34:04.249689 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.241629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:40699 (local address 127.31.57.254:34343)
0809 19:34:04.241708 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.241710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.241910 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:34:04.242396 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.242398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.242411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.242457 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.243313 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.243839 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.244770 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.244965 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.245035 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.246948 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.246949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.246949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:04.246957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.248342 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.248781 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.248782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.248783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.248844 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.249082 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.249083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.249083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.249195 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.249266 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.249518 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.249604 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:34:04.249830 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.241587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34343 (local address 127.31.57.194:40699)
0809 19:34:04.241800 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.241830 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.242283 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.242535 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.242538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.243057 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:04.243211 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.243214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.243929 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.243930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.244663 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.244665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.245043 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.245044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.245152 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.245646 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:04.245653 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:04.246825 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:04.248424 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.248431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.248433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.248701 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.248902 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.248903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.248904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.249006 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.249258 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:04.249265 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:04.249393 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.249495 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.249633 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260809 19:34:04.250808 27542 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34343
I20260809 19:34:04.250911 27542 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:04.251076 27542 heartbeater.cc:511] Master 127.31.57.254:34343 requested a full tablet report, sending...
I20260809 19:34:04.251520 26986 ts_manager.cc:194] Registered new tserver with Master: 67d5228a75914e49ae78a486ba67ef23 (127.31.57.194:44983)
I20260809 19:34:04.251977 26986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:40699
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:34:04.271977 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:04.350284 27549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.350445 27549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.350466 27549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.351529 27549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.351567 27549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.351593 27549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:04.352967 27549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.353130 27549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.353291 27549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.358129 27563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.358161 27556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.358436 27549 server_base.cc:1034] running on GCE node
W20260809 19:34:04.358132 27558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.358798 27549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.359040 27549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.360208 27549 hybrid_clock.cc:648] HybridClock initialized: now 1786304044360196 us; error 40 us; skew 500 ppm
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:04.366454 27549 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:34:04.366853 27549 webserver.cc:460] Webserver started at http://127.31.57.195:34105/ using document root <none> and password file <none>
I20260809 19:34:04.367115 27549 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.367185 27549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.367341 27549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.368427 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "1d5be72d10df4f6d962296ca498ff98b"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "8ed54d7ec3395246bb83f7edc5d073e4"
server_key_iv: "01970b527c6fc25d926af1cdd9aad278"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.368762 27549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5be72d10df4f6d962296ca498ff98b"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "8ed54d7ec3395246bb83f7edc5d073e4"
server_key_iv: "01970b527c6fc25d926af1cdd9aad278"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.370062 27549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:04.370780 27572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.370971 27549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:04.371047 27549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "1d5be72d10df4f6d962296ca498ff98b"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "8ed54d7ec3395246bb83f7edc5d073e4"
server_key_iv: "01970b527c6fc25d926af1cdd9aad278"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.371129 27549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:04.382335 27549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:04.384166 27549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:04.384315 27549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:04.384567 27549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:04.384884 27549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:04.384918 27549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.384938 27549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:04.384984 27549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.390440 27549 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:41921
I20260809 19:34:04.390502 27766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:41921 every 8 connection(s)
I20260809 19:34:04.390787 27549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:34:04.396348 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 27549
Aug 09 19:34:04 dist-test-slave-x4qh krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304044, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:04.396469 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:04.396656 31975 external_mini_cluster.cc:1428] Setting key a4ff6754e913786c91a9ddc7effa59ce
I20260809 19:34:04.399711 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.391495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:56037 (local address 127.31.57.254:34343)
0809 19:34:04.391554 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.391556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.391778 (+ 222us) server_negotiation.cc:407] Connection header received
0809 19:34:04.392239 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.392240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.392253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.392295 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.393162 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.393648 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.394642 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.394841 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.394910 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.396913 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.396914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.396915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:04.396922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.398389 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.398771 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.398772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.398773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.398827 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.399068 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.399068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.399069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.399179 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.399244 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.399556 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.399630 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:34:04.399893 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.391483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:34343 (local address 127.31.57.195:56037)
0809 19:34:04.391650 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.391697 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.392128 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.392372 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.392374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.392901 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:04.393063 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.393068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.393755 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.393756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.394542 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.394544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.394930 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.394931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.395035 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.395472 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:04.395479 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:04.396825 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:04.398486 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.398492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.398494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.398694 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.398900 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:04.398901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:04.398901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:04.399002 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:04.399255 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:04.399261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:04.399420 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.399534 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.399672 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260809 19:34:04.400357 27768 heartbeater.cc:347] Connected to a master server at 127.31.57.254:34343
I20260809 19:34:04.400449 27768 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:04.400578 27768 heartbeater.cc:511] Master 127.31.57.254:34343 requested a full tablet report, sending...
I20260809 19:34:04.400898 26986 ts_manager.cc:194] Registered new tserver with Master: 1d5be72d10df4f6d962296ca498ff98b (127.31.57.195:41921)
I20260809 19:34:04.401206 26986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:56037
I20260809 19:34:04.408483 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:04.415077 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.409871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.31.57.254:34343)
0809 19:34:04.409923 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.409929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.410006 (+ 77us) server_negotiation.cc:407] Connection header received
0809 19:34:04.410042 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.410043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.410051 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.410102 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.410618 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.411043 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.411782 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.411951 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.412002 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.413006 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.413007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.413007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:04.413012 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.414289 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.414546 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.414547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.414547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.414591 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:04.414723 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:04.414723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:04.414723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:04.414831 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.414894 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.414975 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.415025 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:34:04.416498 26986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:04.417902 26986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:04.424101 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.420157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.31.57.194:44983)
0809 19:34:04.420356 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.420360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.420441 (+ 81us) server_negotiation.cc:407] Connection header received
0809 19:34:04.420508 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.420510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.420524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.420607 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:04.420770 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.421666 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423151 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.423722 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.423871 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.423994 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.424030 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260809 19:34:04.424089 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.419937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:44983 (local address 127.0.0.1:55254)
0809 19:34:04.420372 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.420381 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.420451 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.420621 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.420622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.420625 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:04.420679 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.420681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.421814 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.421815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.423050 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.423052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.423823 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.423898 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.423910 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.423976 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.424024 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":46,"thread_start_us":46,"threads_started":1}
I20260809 19:34:04.424253 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.420152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.31.57.195:41921)
0809 19:34:04.420319 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.420323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.420334 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:34:04.420367 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.420369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.420384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.420482 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:04.420728 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.421890 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423102 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.423742 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423810 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.423859 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.424131 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.424168 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260809 19:34:04.424304 27789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.419919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:41921 (local address 127.0.0.1:45278)
0809 19:34:04.420185 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.420223 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.420287 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.420502 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.420505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.420507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:04.420574 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.420576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.421998 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.421999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.423005 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.423008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.423895 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.423971 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.423975 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.424026 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.424068 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":59,"threads_started":1}
I20260809 19:34:04.424528 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.420006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.31.57.193:44523)
0809 19:34:04.420626 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.420630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.420640 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:34:04.420678 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.420680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.420695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.420769 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:04.420963 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.422069 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.423301 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.423989 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.424045 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.424091 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.424363 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.424441 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":100,"threads_started":1}
I20260809 19:34:04.424674 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.419920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:44523 (local address 127.0.0.1:50094)
0809 19:34:04.420190 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.420224 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.420286 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.420857 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.420858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.420860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:04.420893 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.420894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.422176 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.422177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.423223 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.423224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.424125 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.424126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.424176 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.424179 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.424224 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.424257 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":65,"threads_started":1}
I20260809 19:34:04.425132 27421 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad3890036a74a36aba91d0e06901451 (DEFAULT_TABLE table=test-table [id=f632e6c76bd24877bd505c23b2be009c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:04.425132 27658 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad3890036a74a36aba91d0e06901451 (DEFAULT_TABLE table=test-table [id=f632e6c76bd24877bd505c23b2be009c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:04.425465 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad3890036a74a36aba91d0e06901451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:04.425465 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad3890036a74a36aba91d0e06901451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:04.425386 27191 tablet_service.cc:1467] Processing CreateTablet for tablet 4ad3890036a74a36aba91d0e06901451 (DEFAULT_TABLE table=test-table [id=f632e6c76bd24877bd505c23b2be009c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:04.425714 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ad3890036a74a36aba91d0e06901451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:04.428205 27799 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: Bootstrap starting.
I20260809 19:34:04.428908 27799 tablet_bootstrap.cc:654] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:04.428936 27802 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Bootstrap starting.
I20260809 19:34:04.429414 27799 log.cc:824] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:04.429481 27802 tablet_bootstrap.cc:654] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:04.430042 27802 log.cc:824] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:04.430200 27799 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: No bootstrap required, opened a new log
I20260809 19:34:04.430255 27799 ts_tablet_manager.cc:1397] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:04.430387 27800 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Bootstrap starting.
I20260809 19:34:04.430866 27802 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: No bootstrap required, opened a new log
I20260809 19:34:04.430958 27800 tablet_bootstrap.cc:654] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:04.431264 27802 ts_tablet_manager.cc:1397] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:04.431486 27800 log.cc:824] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:04.432209 27800 tablet_bootstrap.cc:492] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: No bootstrap required, opened a new log
I20260809 19:34:04.432298 27800 ts_tablet_manager.cc:1397] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:34:04.432303 27799 raft_consensus.cc:348] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.432485 27799 raft_consensus.cc:374] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:04.432559 27799 raft_consensus.cc:729] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7287b16bb74470d82d3aa442ef4cc31, State: Initialized, Role: FOLLOWER
I20260809 19:34:04.432689 27799 consensus_queue.cc:260] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.432986 27799 ts_tablet_manager.cc:1428] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:34:04.433007 27307 heartbeater.cc:503] Master 127.31.57.254:34343 was elected leader, sending a full tablet report...
I20260809 19:34:04.433295 27802 raft_consensus.cc:348] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.433507 27802 raft_consensus.cc:374] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:04.433584 27802 raft_consensus.cc:729] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d5228a75914e49ae78a486ba67ef23, State: Initialized, Role: FOLLOWER
I20260809 19:34:04.433717 27802 consensus_queue.cc:260] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.433990 27542 heartbeater.cc:503] Master 127.31.57.254:34343 was elected leader, sending a full tablet report...
I20260809 19:34:04.433983 27802 ts_tablet_manager.cc:1428] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:34:04.434242 27800 raft_consensus.cc:348] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.434494 27800 raft_consensus.cc:374] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:04.434542 27800 raft_consensus.cc:729] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5be72d10df4f6d962296ca498ff98b, State: Initialized, Role: FOLLOWER
I20260809 19:34:04.434720 27800 consensus_queue.cc:260] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.434995 27800 ts_tablet_manager.cc:1428] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:34:04.435015 27768 heartbeater.cc:503] Master 127.31.57.254:34343 was elected leader, sending a full tablet report...
W20260809 19:34:04.491495 27543 tablet.cc:2369] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:04.550369 27811 raft_consensus.cc:482] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:04.550490 27811 raft_consensus.cc:504] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.550927 27811 leader_election.cc:288] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5be72d10df4f6d962296ca498ff98b (127.31.57.195:41921), 67d5228a75914e49ae78a486ba67ef23 (127.31.57.194:44983)
I20260809 19:34:04.554880 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.551234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:39045 (local address 127.31.57.194:44983)
0809 19:34:04.551287 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.551289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.551298 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:04.551327 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.551328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.551343 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.551470 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:04.551673 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.552454 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.553945 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.554502 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.554590 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.554599 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.554770 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.554812 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:34:04.554877 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.551125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:44983 (local address 127.31.57.193:39045)
0809 19:34:04.551183 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.551194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.551239 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.551481 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.551482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.551484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:04.551589 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.551591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.552535 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.552536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.553853 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.553857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.554599 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.554600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.554683 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.554687 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.554745 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.554799 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":117}
I20260809 19:34:04.554962 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.551125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:41921 (local address 127.31.57.193:51039)
0809 19:34:04.551327 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.551335 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.551456 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.551602 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.551603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.551605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:04.551666 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.551667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.552535 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.552536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.553853 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.553857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.554638 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.554639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.554705 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.554709 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.554782 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.554819 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":103,"thread_start_us":147,"threads_started":1}
I20260809 19:34:04.555017 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.551232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:51039 (local address 127.31.57.195:41921)
0809 19:34:04.551287 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.551289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.551457 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:34:04.551513 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.551514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.551523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.551576 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:04.551751 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.552447 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.553945 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.554537 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.554591 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.554599 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.554904 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.554947 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:34:04.555261 27468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad3890036a74a36aba91d0e06901451" candidate_uuid: "e7287b16bb74470d82d3aa442ef4cc31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d5228a75914e49ae78a486ba67ef23" is_pre_election: true
I20260809 19:34:04.555284 27690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad3890036a74a36aba91d0e06901451" candidate_uuid: "e7287b16bb74470d82d3aa442ef4cc31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5be72d10df4f6d962296ca498ff98b" is_pre_election: true
I20260809 19:34:04.555454 27690 raft_consensus.cc:2454] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7287b16bb74470d82d3aa442ef4cc31 in term 0.
I20260809 19:34:04.555454 27468 raft_consensus.cc:2454] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7287b16bb74470d82d3aa442ef4cc31 in term 0.
I20260809 19:34:04.555677 27116 leader_election.cc:302] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5be72d10df4f6d962296ca498ff98b, e7287b16bb74470d82d3aa442ef4cc31; no voters:
I20260809 19:34:04.555835 27811 raft_consensus.cc:2781] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:04.555886 27811 raft_consensus.cc:482] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:04.555912 27811 raft_consensus.cc:3037] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:04.556533 27811 raft_consensus.cc:504] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.556685 27811 leader_election.cc:288] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5be72d10df4f6d962296ca498ff98b (127.31.57.195:41921), 67d5228a75914e49ae78a486ba67ef23 (127.31.57.194:44983)
I20260809 19:34:04.556872 27468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad3890036a74a36aba91d0e06901451" candidate_uuid: "e7287b16bb74470d82d3aa442ef4cc31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d5228a75914e49ae78a486ba67ef23"
I20260809 19:34:04.556872 27690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ad3890036a74a36aba91d0e06901451" candidate_uuid: "e7287b16bb74470d82d3aa442ef4cc31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5be72d10df4f6d962296ca498ff98b"
I20260809 19:34:04.556946 27690 raft_consensus.cc:3037] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:04.556946 27468 raft_consensus.cc:3037] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:04.557822 27468 raft_consensus.cc:2454] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7287b16bb74470d82d3aa442ef4cc31 in term 1.
I20260809 19:34:04.557823 27690 raft_consensus.cc:2454] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7287b16bb74470d82d3aa442ef4cc31 in term 1.
I20260809 19:34:04.557999 27116 leader_election.cc:302] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5be72d10df4f6d962296ca498ff98b, e7287b16bb74470d82d3aa442ef4cc31; no voters:
I20260809 19:34:04.558105 27811 raft_consensus.cc:2781] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:04.558233 27811 raft_consensus.cc:686] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 1 LEADER]: Becoming Leader. State: Replica: e7287b16bb74470d82d3aa442ef4cc31, State: Running, Role: LEADER
I20260809 19:34:04.558336 27811 consensus_queue.cc:237] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } }
I20260809 19:34:04.559139 26986 catalog_manager.cc:5495] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7287b16bb74470d82d3aa442ef4cc31 (127.31.57.193). New cstate: current_term: 1 leader_uuid: "e7287b16bb74470d82d3aa442ef4cc31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7287b16bb74470d82d3aa442ef4cc31" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 44523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 } health_report { overall_health: UNKNOWN } } }
I20260809 19:34:04.569602 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.567758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.31.57.193:44523)
0809 19:34:04.567805 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.567806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.567813 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:04.567857 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.567858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.567870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.567925 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:04.568023 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.568415 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.569029 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.569187 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.569241 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.569363 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:04.569445 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:04.569500 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.569527 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.569543 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:34:04.571111 27468 raft_consensus.cc:1261] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 1 FOLLOWER]: Refusing update from remote peer e7287b16bb74470d82d3aa442ef4cc31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:04.571138 27690 raft_consensus.cc:1261] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 1 FOLLOWER]: Refusing update from remote peer e7287b16bb74470d82d3aa442ef4cc31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:04.571439 27820 consensus_queue.cc:1035] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5be72d10df4f6d962296ca498ff98b" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 41921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:04.571563 27811 consensus_queue.cc:1035] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d5228a75914e49ae78a486ba67ef23" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:04.573711 27824 mvcc.cc:205] Tried to move back new op lower bound from 7316701366560997376 to 7316701366511489024. Current Snapshot: MvccSnapshot[applied={T|T < 7316701366560997376}]
I20260809 19:34:04.574373 27827 mvcc.cc:205] Tried to move back new op lower bound from 7316701366560997376 to 7316701366511489024. Current Snapshot: MvccSnapshot[applied={T|T < 7316701366560997376}]
I20260809 19:34:04.574378 27826 mvcc.cc:205] Tried to move back new op lower bound from 7316701366560997376 to 7316701366511489024. Current Snapshot: MvccSnapshot[applied={T|T < 7316701366560997376}]
I20260809 19:34:04.576051 26986 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:04.576161 26986 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:04.576840 26986 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96764a0933a74698bead3de46e1d7680: Sending DeleteTablet for 3 replicas of tablet 4ad3890036a74a36aba91d0e06901451
I20260809 19:34:04.577124 27658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad3890036a74a36aba91d0e06901451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:04 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:45278
I20260809 19:34:04.577142 27191 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad3890036a74a36aba91d0e06901451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:04 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:50094
I20260809 19:34:04.577348 27421 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ad3890036a74a36aba91d0e06901451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:04 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:55254
I20260809 19:34:04.577570 27838 tablet_replica.cc:330] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: stopping tablet replica
I20260809 19:34:04.577564 27843 tablet_replica.cc:330] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: stopping tablet replica
I20260809 19:34:04.577658 27838 raft_consensus.cc:2229] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:04.577658 27843 raft_consensus.cc:2229] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:04.577750 27843 raft_consensus.cc:2258] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:04.577754 27838 raft_consensus.cc:2258] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:04.577795 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 27083
I20260809 19:34:04.577917 27842 tablet_replica.cc:330] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31: stopping tablet replica
I20260809 19:34:04.577991 27842 raft_consensus.cc:2229] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:04.578064 27842 raft_consensus.cc:2258] T 4ad3890036a74a36aba91d0e06901451 P e7287b16bb74470d82d3aa442ef4cc31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:04.578923 27838 ts_tablet_manager.cc:1905] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:04.580888 27838 ts_tablet_manager.cc:1918] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:04.580948 27838 log.cc:1196] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/4ad3890036a74a36aba91d0e06901451
I20260809 19:34:04.581179 27838 ts_tablet_manager.cc:1939] T 4ad3890036a74a36aba91d0e06901451 P 1d5be72d10df4f6d962296ca498ff98b: Deleting consensus metadata
I20260809 19:34:04.582773 27843 ts_tablet_manager.cc:1905] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:04.582851 26958 catalog_manager.cc:4841] TS 1d5be72d10df4f6d962296ca498ff98b (127.31.57.195:41921): tablet 4ad3890036a74a36aba91d0e06901451 (table test-table [id=f632e6c76bd24877bd505c23b2be009c]) successfully deleted
I20260809 19:34:04.585013 27843 ts_tablet_manager.cc:1918] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:04.585096 27843 log.cc:1196] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/4ad3890036a74a36aba91d0e06901451
I20260809 19:34:04.585424 27843 ts_tablet_manager.cc:1939] T 4ad3890036a74a36aba91d0e06901451 P 67d5228a75914e49ae78a486ba67ef23: Deleting consensus metadata
I20260809 19:34:04.585877 26957 catalog_manager.cc:4841] TS 67d5228a75914e49ae78a486ba67ef23 (127.31.57.194:44983): tablet 4ad3890036a74a36aba91d0e06901451 (table test-table [id=f632e6c76bd24877bd505c23b2be009c]) successfully deleted
W20260809 19:34:04.590137 26954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:44523 (error 108)
I20260809 19:34:04.590348 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 27315
I20260809 19:34:04.595098 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.590471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:44523 (local address 127.0.0.1:50122)
0809 19:34:04.595012 (+ 4541us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.595058 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:44523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4519}
W20260809 19:34:04.595257 26954 catalog_manager.cc:4584] TS e7287b16bb74470d82d3aa442ef4cc31 (127.31.57.193:44523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ad3890036a74a36aba91d0e06901451: Network error: Client connection negotiation failed: client connection to 127.31.57.193:44523: connect: Connection refused (error 111)
I20260809 19:34:04.601159 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 27549
I20260809 19:34:04.609750 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 26927
2026-08-09T19:34:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2144 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T19:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:04Z Disabled control of system clock
I20260809 19:34:04.642644 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:36113
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:44051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:36113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:04.719245 27849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.719493 27849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.719527 27849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.720544 27849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:04.720606 27849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.720625 27849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.720643 27849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:04.720671 27849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:04.721896 27849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:36113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:36113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.722148 27849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.722425 27849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.727036 27857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.727069 27861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.727061 27858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.727412 27849 server_base.cc:1034] running on GCE node
I20260809 19:34:04.727648 27849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.727916 27849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.729058 27849 hybrid_clock.cc:648] HybridClock initialized: now 1786304044729046 us; error 27 us; skew 500 ppm
I20260809 19:34:04.730293 27849 webserver.cc:460] Webserver started at http://127.31.57.254:42175/ using document root <none> and password file <none>
I20260809 19:34:04.730532 27849 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.730576 27849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.730733 27849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.731829 27849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "5caaa2f33f6847b99408ee4d89f0a04f"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "d60319b8d024d14d321ae1c0c17d2399"
server_key_iv: "e925f13846471a381e50bc1a55a90a90"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.732182 27849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "5caaa2f33f6847b99408ee4d89f0a04f"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "d60319b8d024d14d321ae1c0c17d2399"
server_key_iv: "e925f13846471a381e50bc1a55a90a90"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.733453 27849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:04.734164 27867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.734338 27849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:04.734427 27849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "5caaa2f33f6847b99408ee4d89f0a04f"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "d60319b8d024d14d321ae1c0c17d2399"
server_key_iv: "e925f13846471a381e50bc1a55a90a90"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.734511 27849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:04.743614 27849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:04.743866 27849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:04.744009 27849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:04.747997 27849 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:36113
I20260809 19:34:04.748026 27963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:36113 every 8 connection(s)
I20260809 19:34:04.748379 27849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:04.749256 27964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:04.751222 27964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: Bootstrap starting.
I20260809 19:34:04.751838 27964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:04.752094 27964 log.cc:824] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:04.752681 27964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: No bootstrap required, opened a new log
I20260809 19:34:04.754230 27964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } }
I20260809 19:34:04.754364 27964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:04.754385 27964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5caaa2f33f6847b99408ee4d89f0a04f, State: Initialized, Role: FOLLOWER
I20260809 19:34:04.754473 27964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } }
I20260809 19:34:04.754518 27964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:04.754535 27964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:04.754565 27964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:04.754987 27964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } }
I20260809 19:34:04.755081 27964 leader_election.cc:302] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5caaa2f33f6847b99408ee4d89f0a04f; no voters:
I20260809 19:34:04.755208 27964 leader_election.cc:288] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:04.755275 27968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:04.755460 27968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [term 1 LEADER]: Becoming Leader. State: Replica: 5caaa2f33f6847b99408ee4d89f0a04f, State: Running, Role: LEADER
I20260809 19:34:04.755488 27964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:04.755573 27968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } }
I20260809 19:34:04.755931 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5caaa2f33f6847b99408ee4d89f0a04f. Latest consensus state: current_term: 1 leader_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } } }
I20260809 19:34:04.756000 27969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5caaa2f33f6847b99408ee4d89f0a04f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36113 } } }
I20260809 19:34:04.756022 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:04.756047 27969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:04.756212 27976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:04.756852 27976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:04.757009 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 27849
I20260809 19:34:04.757099 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:04.757311 31975 external_mini_cluster.cc:1428] Setting key fc293392fa0efb671830cbeaeb5709b3
I20260809 19:34:04.758417 27976 catalog_manager.cc:1347] Generated new cluster ID: 7539d84560064167b67d10803e2af711
I20260809 19:34:04.758469 27976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:04.762991 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.757763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.31.57.254:36113)
0809 19:34:04.757939 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.757947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.758016 (+ 69us) server_negotiation.cc:407] Connection header received
0809 19:34:04.758186 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.758193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.758257 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.758628 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.758725 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.759953 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.760922 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.761549 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.761672 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.762294 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.762298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.762298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:04.762496 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.762536 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:04.762550 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.762620 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.762684 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.762757 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260809 19:34:04.769212 27976 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:04.769799 27976 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:04.777011 27976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: Generated new TSK 0
I20260809 19:34:04.777161 27976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:04.778774 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--builtin_ntp_servers=127.31.57.212:44051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:04.856448 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.856644 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.856678 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.857748 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.857808 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.857856 28004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:04.858981 28004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.859225 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.859537 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.864100 28018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.864112 28015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.864377 28004 server_base.cc:1034] running on GCE node
W20260809 19:34:04.864288 28016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.864640 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.864889 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.866067 28004 hybrid_clock.cc:648] HybridClock initialized: now 1786304044866038 us; error 47 us; skew 500 ppm
I20260809 19:34:04.867334 28004 webserver.cc:460] Webserver started at http://127.31.57.193:44585/ using document root <none> and password file <none>
I20260809 19:34:04.867581 28004 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.867668 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.867823 28004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.868775 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "787cece96584480c8a636a879d243092"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5f56e19d5786eae52be010524f8a99"
server_key_iv: "4e50e901f0c9d2486907aa4b0c59a86b"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.869120 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "787cece96584480c8a636a879d243092"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5f56e19d5786eae52be010524f8a99"
server_key_iv: "4e50e901f0c9d2486907aa4b0c59a86b"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.870362 28004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:34:04.871054 28025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.871201 28004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:04.871276 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "787cece96584480c8a636a879d243092"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "5b5f56e19d5786eae52be010524f8a99"
server_key_iv: "4e50e901f0c9d2486907aa4b0c59a86b"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.871335 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:04.883493 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:04.883752 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:04.883872 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:04.884038 28004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:04.884289 28004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:04.884320 28004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.884353 28004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:04.884367 28004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.890267 28004 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:43743
I20260809 19:34:04.890318 28207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:43743 every 8 connection(s)
I20260809 19:34:04.890607 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:34:04.894002 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 28004
I20260809 19:34:04.894147 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:04.894368 31975 external_mini_cluster.cc:1428] Setting key 71757ccbb77dacc0cf01ca3a7865a0b3
I20260809 19:34:04.895448 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--builtin_ntp_servers=127.31.57.212:44051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:34:04.896164 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.891260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:51129 (local address 127.31.57.254:36113)
0809 19:34:04.891305 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:04.891307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:04.891567 (+ 260us) server_negotiation.cc:407] Connection header received
0809 19:34:04.892119 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:04.892121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:04.892137 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:04.892205 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:04.892862 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.893341 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.894613 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.894817 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.894913 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.895885 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:04.895886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:04.895886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:04.895890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:04.895906 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:04.895910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:04.895972 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:04.896044 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 19:34:04.896089 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:34:04.896404 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:04.891250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36113 (local address 127.31.57.193:51129)
0809 19:34:04.891437 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:04.891481 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:04.891999 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:04.892270 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:04.892274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:04.892283 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:04.892745 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.892750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.893441 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.893443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.894517 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:04.894520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:04.894928 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:04.894929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:04.895117 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:04.895742 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:04.895742 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:04.895767 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:04.895791 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:04.895973 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:04.895980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:04.895982 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:04.896048 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:34:04.896163 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260809 19:34:04.896970 28212 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36113
I20260809 19:34:04.897078 28212 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:04.897254 28212 heartbeater.cc:511] Master 127.31.57.254:36113 requested a full tablet report, sending...
I20260809 19:34:04.897857 27898 ts_manager.cc:194] Registered new tserver with Master: 787cece96584480c8a636a879d243092 (127.31.57.193:43743)
I20260809 19:34:04.898638 27898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.193:51129
W20260809 19:34:04.975643 28216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:04.975839 28216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:04.975860 28216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:04.976824 28216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:04.976861 28216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:04.976884 28216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:04.978036 28216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:04.978214 28216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:04.978434 28216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:04.983232 28225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:04.983238 28228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.983268 28216 server_base.cc:1034] running on GCE node
W20260809 19:34:04.983253 28224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:04.983672 28216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:04.983896 28216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:04.985033 28216 hybrid_clock.cc:648] HybridClock initialized: now 1786304044985010 us; error 40 us; skew 500 ppm
I20260809 19:34:04.986177 28216 webserver.cc:460] Webserver started at http://127.31.57.194:38077/ using document root <none> and password file <none>
I20260809 19:34:04.986423 28216 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:04.986469 28216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:04.986632 28216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:04.987679 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "26dc5e8eca7a4e61b3087e2286eb749a"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "a267e197eb7f5461f111afc876d2c97c"
server_key_iv: "09ed2d164536fdf39590e1e618662843"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.988008 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "26dc5e8eca7a4e61b3087e2286eb749a"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "a267e197eb7f5461f111afc876d2c97c"
server_key_iv: "09ed2d164536fdf39590e1e618662843"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.989140 28216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:04.989805 28239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:04.990005 28216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:04.990108 28216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "26dc5e8eca7a4e61b3087e2286eb749a"
format_stamp: "Formatted at 2026-08-09 19:34:04 on dist-test-slave-x4qh"
server_key: "a267e197eb7f5461f111afc876d2c97c"
server_key_iv: "09ed2d164536fdf39590e1e618662843"
server_key_version: "encryptionkey@0"
I20260809 19:34:04.990186 28216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:05.004626 28216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:05.004878 28216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:05.005028 28216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:05.005227 28216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:05.005527 28216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:05.005601 28216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:05.005642 28216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:05.005671 28216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:05.011309 28216 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:41021
I20260809 19:34:05.011343 28420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:41021 every 8 connection(s)
I20260809 19:34:05.011732 28216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:34:05.016351 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.012421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:48587 (local address 127.31.57.254:36113)
0809 19:34:05.012488 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.012490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.012686 (+ 196us) server_negotiation.cc:407] Connection header received
0809 19:34:05.013095 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.013096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.013105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.013158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:05.013747 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.014218 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.015107 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.015282 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.015333 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.016160 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:05.016161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:05.016162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:05.016166 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:05.016178 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:05.016181 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:05.016220 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.016276 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.016296 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:34:05.016538 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.012387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36113 (local address 127.31.57.194:48587)
0809 19:34:05.012557 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.012594 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.013013 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.013204 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.013207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.013214 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:05.013674 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.013678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.014303 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.014303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.015032 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.015034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.015357 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.015358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.015509 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.016039 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:05.016039 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:05.016065 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:05.016086 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:05.016222 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:05.016224 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:05.016226 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.016268 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.016345 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260809 19:34:05.017060 28421 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36113
I20260809 19:34:05.017161 28421 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:05.017329 28421 heartbeater.cc:511] Master 127.31.57.254:36113 requested a full tablet report, sending...
I20260809 19:34:05.017705 27898 ts_manager.cc:194] Registered new tserver with Master: 26dc5e8eca7a4e61b3087e2286eb749a (127.31.57.194:41021)
I20260809 19:34:05.018038 27898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.194:48587
I20260809 19:34:05.021910 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 28216
I20260809 19:34:05.022013 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:05.022212 31975 external_mini_cluster.cc:1428] Setting key 884dcbbdc1557e4bdb3b85e25cf8e356
I20260809 19:34:05.023123 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--builtin_ntp_servers=127.31.57.212:44051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:05.102756 28427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:05.102955 28427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:05.102974 28427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:05.104008 28427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:05.104053 28427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:05.104076 28427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:05.105223 28427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:05.105415 28427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:05.105635 28427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:05.110323 28439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:05.110332 28436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:05.110312 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:05.110517 28427 server_base.cc:1034] running on GCE node
I20260809 19:34:05.110755 28427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:05.110989 28427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:05.112130 28427 hybrid_clock.cc:648] HybridClock initialized: now 1786304045112135 us; error 48 us; skew 500 ppm
I20260809 19:34:05.113293 28427 webserver.cc:460] Webserver started at http://127.31.57.195:40259/ using document root <none> and password file <none>
I20260809 19:34:05.113513 28427 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:05.113554 28427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:05.113683 28427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:05.114676 28427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "1b60ba9320fc448ba5639fa9a220d5dd"
format_stamp: "Formatted at 2026-08-09 19:34:05 on dist-test-slave-x4qh"
server_key: "49feda53c4ddfcf5b4e647440a7828c6"
server_key_iv: "6387209c6ecf4dae3d88b8931e04efa6"
server_key_version: "encryptionkey@0"
I20260809 19:34:05.115032 28427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "1b60ba9320fc448ba5639fa9a220d5dd"
format_stamp: "Formatted at 2026-08-09 19:34:05 on dist-test-slave-x4qh"
server_key: "49feda53c4ddfcf5b4e647440a7828c6"
server_key_iv: "6387209c6ecf4dae3d88b8931e04efa6"
server_key_version: "encryptionkey@0"
I20260809 19:34:05.116310 28427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:34:05.116961 28448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:05.117122 28427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:05.117244 28427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "1b60ba9320fc448ba5639fa9a220d5dd"
format_stamp: "Formatted at 2026-08-09 19:34:05 on dist-test-slave-x4qh"
server_key: "49feda53c4ddfcf5b4e647440a7828c6"
server_key_iv: "6387209c6ecf4dae3d88b8931e04efa6"
server_key_version: "encryptionkey@0"
I20260809 19:34:05.117323 28427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:05.137336 28427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:05.137617 28427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:05.137739 28427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:05.137944 28427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:05.138262 28427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:05.138321 28427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:05.138350 28427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:05.138373 28427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:05.143844 28427 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:46015
I20260809 19:34:05.143893 28649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:46015 every 8 connection(s)
I20260809 19:34:05.144158 28427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:34:05.148423 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.144742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:33819 (local address 127.31.57.254:36113)
0809 19:34:05.144795 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.144798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.145016 (+ 218us) server_negotiation.cc:407] Connection header received
0809 19:34:05.145424 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.145425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.145435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.145485 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:05.146046 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.146481 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.147310 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.147491 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.147536 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.148176 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:05.148178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:05.148178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:05.148182 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:05.148195 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:05.148199 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:05.148242 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.148287 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.148309 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:05.148540 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 28427
I20260809 19:34:05.148595 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.144748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36113 (local address 127.31.57.195:33819)
0809 19:34:05.144912 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.144942 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.145347 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.145542 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.145544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.145552 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:05.145986 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.145989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.146555 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.146555 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.147241 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.147243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.147557 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.147558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.147658 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.148058 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:05.148058 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:05.148081 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:05.148101 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:05.148242 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:05.148244 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:05.148246 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.148289 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.148356 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":91,"threads_started":1}
I20260809 19:34:05.148629 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:05.148836 31975 external_mini_cluster.cc:1428] Setting key 63d4f079eef7d6df9ecc6d6e205202ec
I20260809 19:34:05.149189 28650 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36113
I20260809 19:34:05.149308 28650 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:05.149502 28650 heartbeater.cc:511] Master 127.31.57.254:36113 requested a full tablet report, sending...
I20260809 19:34:05.149875 27898 ts_manager.cc:194] Registered new tserver with Master: 1b60ba9320fc448ba5639fa9a220d5dd (127.31.57.195:46015)
I20260809 19:34:05.150213 27898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.195:33819
I20260809 19:34:05.160601 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:05.164443 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.161905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.31.57.254:36113)
0809 19:34:05.161972 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.161973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.162037 (+ 64us) server_negotiation.cc:407] Connection header received
0809 19:34:05.162073 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.162073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.162082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.162127 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:05.162269 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.162681 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.163333 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.163540 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.163634 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.164162 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:05.164163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:05.164164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:05.164167 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:05.164178 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:05.164181 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:05.164223 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.164292 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.164310 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:34:05.165678 27898 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:05.167052 27898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:05.173075 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43743 (local address 127.0.0.1:35628)
0809 19:34:05.169554 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.169569 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.169616 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.170020 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.170020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.170022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:05.170056 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.170057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.171220 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.171221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.172022 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.172026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.172789 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.172866 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.172879 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.172958 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.173004 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":58,"threads_started":1}
I20260809 19:34:05.173080 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.31.57.193:43743)
0809 19:34:05.169809 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.169812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.169820 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:05.169852 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.169853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.169864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.169963 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:05.170133 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.171148 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172125 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.172689 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172780 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.172867 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.172972 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.173008 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260809 19:34:05.173254 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:41021 (local address 127.0.0.1:37560)
0809 19:34:05.169532 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.169543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.169605 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.169947 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.169948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.169950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:05.169987 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.169988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.170934 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.170935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.172161 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.172163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.173052 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.173053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.173117 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.173120 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.173147 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.173181 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":28,"thread_start_us":82,"threads_started":1}
I20260809 19:34:05.173395 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44194 (local address 127.31.57.195:46015)
0809 19:34:05.169293 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.169298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.169307 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:05.169340 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.169341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.169354 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.169429 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:05.169945 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.171001 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172261 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.172953 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172999 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.173070 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.173295 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.173334 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260809 19:34:05.173383 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:46015 (local address 127.0.0.1:44194)
0809 19:34:05.169148 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.169189 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.169258 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.169449 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.169451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.169453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:05.169529 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.169532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.171082 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.171083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.172157 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.172160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.173122 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.173123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.173171 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.173173 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.173213 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.173240 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":23,"thread_start_us":52,"threads_started":1}
I20260809 19:34:05.173578 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.169329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.31.57.194:41021)
0809 19:34:05.169738 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.169743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.169752 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:05.169803 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.169806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.169821 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.169946 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:05.170062 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.170849 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.172255 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.172953 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.173350 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.173438 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.173464 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.173500 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":86,"threads_started":1}
I20260809 19:34:05.174252 28311 tablet_service.cc:1467] Processing CreateTablet for tablet e99d912a049e4c418d91e700d45888e2 (DEFAULT_TABLE table=test-table [id=a5e05059478143e99aeea23f912358ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:05.174254 28103 tablet_service.cc:1467] Processing CreateTablet for tablet e99d912a049e4c418d91e700d45888e2 (DEFAULT_TABLE table=test-table [id=a5e05059478143e99aeea23f912358ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:05.174343 28537 tablet_service.cc:1467] Processing CreateTablet for tablet e99d912a049e4c418d91e700d45888e2 (DEFAULT_TABLE table=test-table [id=a5e05059478143e99aeea23f912358ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:05.174595 28311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99d912a049e4c418d91e700d45888e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:05.174616 28103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99d912a049e4c418d91e700d45888e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:05.174685 28537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e99d912a049e4c418d91e700d45888e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:05.176986 28673 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Bootstrap starting.
I20260809 19:34:05.177603 28673 tablet_bootstrap.cc:654] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:05.177588 28674 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Bootstrap starting.
I20260809 19:34:05.177920 28673 log.cc:824] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:05.178231 28674 tablet_bootstrap.cc:654] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:05.178521 28674 log.cc:824] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:05.178761 28673 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: No bootstrap required, opened a new log
I20260809 19:34:05.178867 28677 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: Bootstrap starting.
I20260809 19:34:05.178898 28673 ts_tablet_manager.cc:1397] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:05.179169 28674 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: No bootstrap required, opened a new log
I20260809 19:34:05.179247 28674 ts_tablet_manager.cc:1397] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:05.179575 28677 tablet_bootstrap.cc:654] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:05.179927 28677 log.cc:824] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:05.180720 28677 tablet_bootstrap.cc:492] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: No bootstrap required, opened a new log
I20260809 19:34:05.180804 28677 ts_tablet_manager.cc:1397] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:05.181397 28674 raft_consensus.cc:348] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.181545 28674 raft_consensus.cc:374] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:05.181599 28674 raft_consensus.cc:729] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26dc5e8eca7a4e61b3087e2286eb749a, State: Initialized, Role: FOLLOWER
I20260809 19:34:05.181603 28673 raft_consensus.cc:348] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.181789 28673 raft_consensus.cc:374] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:05.181715 28674 consensus_queue.cc:260] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.181825 28673 raft_consensus.cc:729] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b60ba9320fc448ba5639fa9a220d5dd, State: Initialized, Role: FOLLOWER
I20260809 19:34:05.181998 28674 ts_tablet_manager.cc:1428] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:34:05.181977 28673 consensus_queue.cc:260] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.182108 28421 heartbeater.cc:503] Master 127.31.57.254:36113 was elected leader, sending a full tablet report...
I20260809 19:34:05.182221 28673 ts_tablet_manager.cc:1428] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:05.182250 28650 heartbeater.cc:503] Master 127.31.57.254:36113 was elected leader, sending a full tablet report...
I20260809 19:34:05.183362 28677 raft_consensus.cc:348] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.183535 28677 raft_consensus.cc:374] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:05.183559 28677 raft_consensus.cc:729] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787cece96584480c8a636a879d243092, State: Initialized, Role: FOLLOWER
I20260809 19:34:05.183660 28677 consensus_queue.cc:260] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.183861 28677 ts_tablet_manager.cc:1428] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 19:34:05.183950 28212 heartbeater.cc:503] Master 127.31.57.254:36113 was elected leader, sending a full tablet report...
W20260809 19:34:05.262285 28422 tablet.cc:2369] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:05.325086 28686 raft_consensus.cc:482] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:05.325203 28686 raft_consensus.cc:504] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.325516 28686 leader_election.cc:288] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b60ba9320fc448ba5639fa9a220d5dd (127.31.57.195:46015), 787cece96584480c8a636a879d243092 (127.31.57.193:43743)
I20260809 19:34:05.328430 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.325761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:50843 (local address 127.31.57.195:46015)
0809 19:34:05.325821 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.325822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.325874 (+ 52us) server_negotiation.cc:407] Connection header received
0809 19:34:05.325960 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.325961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.325974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.326032 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:05.326212 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.326717 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.327709 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.328076 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.328131 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.328137 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.328314 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.328357 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:34:05.328439 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.325697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:46015 (local address 127.31.57.194:50843)
0809 19:34:05.325793 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.325806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.325876 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.326053 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.326054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.326056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:05.326139 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.326141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.326808 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.326809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.327619 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.327621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.328161 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.328162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.328240 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.328243 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.328287 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.328342 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 19:34:05.328460 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.325804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43743 (local address 127.31.57.194:47733)
0809 19:34:05.325946 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.325956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:05.326019 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:05.326163 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:05.326164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:05.326165 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:05.326252 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.326255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.326858 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.326859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.327630 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:05.327633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.328202 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.328203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:05.328257 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.328260 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:05.328338 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:34:05.328370 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260809 19:34:05.328473 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.325884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:47733 (local address 127.31.57.193:43743)
0809 19:34:05.325926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.325928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.326058 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:34:05.326086 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.326087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.326099 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.326151 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:05.326329 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.326780 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.327751 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.328091 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.328184 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.328191 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.328361 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.328407 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:34:05.328802 28138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99d912a049e4c418d91e700d45888e2" candidate_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787cece96584480c8a636a879d243092" is_pre_election: true
I20260809 19:34:05.328804 28571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99d912a049e4c418d91e700d45888e2" candidate_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" is_pre_election: true
I20260809 19:34:05.328958 28138 raft_consensus.cc:2454] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26dc5e8eca7a4e61b3087e2286eb749a in term 0.
I20260809 19:34:05.328969 28571 raft_consensus.cc:2454] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26dc5e8eca7a4e61b3087e2286eb749a in term 0.
I20260809 19:34:05.329209 28248 leader_election.cc:302] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b60ba9320fc448ba5639fa9a220d5dd, 26dc5e8eca7a4e61b3087e2286eb749a; no voters:
I20260809 19:34:05.329344 28686 raft_consensus.cc:2781] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:05.329401 28686 raft_consensus.cc:482] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:05.329427 28686 raft_consensus.cc:3037] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:05.330101 28686 raft_consensus.cc:504] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.330227 28686 leader_election.cc:288] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [CANDIDATE]: Term 1 election: Requested vote from peers 1b60ba9320fc448ba5639fa9a220d5dd (127.31.57.195:46015), 787cece96584480c8a636a879d243092 (127.31.57.193:43743)
I20260809 19:34:05.330410 28571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99d912a049e4c418d91e700d45888e2" candidate_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b60ba9320fc448ba5639fa9a220d5dd"
I20260809 19:34:05.330435 28138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e99d912a049e4c418d91e700d45888e2" candidate_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787cece96584480c8a636a879d243092"
I20260809 19:34:05.330509 28571 raft_consensus.cc:3037] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:05.330500 28138 raft_consensus.cc:3037] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:05.331261 28138 raft_consensus.cc:2454] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26dc5e8eca7a4e61b3087e2286eb749a in term 1.
I20260809 19:34:05.331292 28571 raft_consensus.cc:2454] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26dc5e8eca7a4e61b3087e2286eb749a in term 1.
I20260809 19:34:05.331451 28248 leader_election.cc:302] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b60ba9320fc448ba5639fa9a220d5dd, 26dc5e8eca7a4e61b3087e2286eb749a; no voters:
I20260809 19:34:05.331574 28686 raft_consensus.cc:2781] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:05.331745 28686 raft_consensus.cc:686] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 1 LEADER]: Becoming Leader. State: Replica: 26dc5e8eca7a4e61b3087e2286eb749a, State: Running, Role: LEADER
I20260809 19:34:05.331856 28686 consensus_queue.cc:237] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } }
I20260809 19:34:05.332650 27896 catalog_manager.cc:5495] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a reported cstate change: term changed from 0 to 1, leader changed from <none> to 26dc5e8eca7a4e61b3087e2286eb749a (127.31.57.194). New cstate: current_term: 1 leader_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26dc5e8eca7a4e61b3087e2286eb749a" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 } health_report { overall_health: UNKNOWN } } }
I20260809 19:34:05.354470 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.352250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.31.57.194:41021)
0809 19:34:05.352305 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:05.352307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:05.352316 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:05.352389 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:05.352395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:05.352409 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:05.352482 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:05.352612 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.353047 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.353738 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:05.353922 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:05.353988 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:05.354155 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:05.354239 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:05.354321 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:05.354360 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:34:05.354377 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:34:05.356053 28571 raft_consensus.cc:1261] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 1 FOLLOWER]: Refusing update from remote peer 26dc5e8eca7a4e61b3087e2286eb749a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:05.356153 28138 raft_consensus.cc:1261] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 1 FOLLOWER]: Refusing update from remote peer 26dc5e8eca7a4e61b3087e2286eb749a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:05.356366 28692 consensus_queue.cc:1035] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b60ba9320fc448ba5639fa9a220d5dd" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:05.356490 28686 consensus_queue.cc:1035] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [LEADER]: Connected to new peer: Peer: permanent_uuid: "787cece96584480c8a636a879d243092" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 43743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:05.358601 28695 mvcc.cc:205] Tried to move back new op lower bound from 7316701369775964160 to 7316701369679855616. Current Snapshot: MvccSnapshot[applied={T|T < 7316701369775964160}]
I20260809 19:34:05.358728 28703 mvcc.cc:205] Tried to move back new op lower bound from 7316701369775964160 to 7316701369679855616. Current Snapshot: MvccSnapshot[applied={T|T < 7316701369775964160}]
I20260809 19:34:05.358950 28700 mvcc.cc:205] Tried to move back new op lower bound from 7316701369775964160 to 7316701369679855616. Current Snapshot: MvccSnapshot[applied={T|T < 7316701369775964160}]
I20260809 19:34:05.360797 27896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:05.360910 27896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:05.361610 27896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5caaa2f33f6847b99408ee4d89f0a04f: Sending DeleteTablet for 3 replicas of tablet e99d912a049e4c418d91e700d45888e2
I20260809 19:34:05.361972 28311 tablet_service.cc:1514] Processing DeleteTablet for tablet e99d912a049e4c418d91e700d45888e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:05 UTC) from {username='slave'} at 127.0.0.1:37560
I20260809 19:34:05.362012 28537 tablet_service.cc:1514] Processing DeleteTablet for tablet e99d912a049e4c418d91e700d45888e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:05 UTC) from {username='slave'} at 127.0.0.1:44194
I20260809 19:34:05.362069 28103 tablet_service.cc:1514] Processing DeleteTablet for tablet e99d912a049e4c418d91e700d45888e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:05 UTC) from {username='slave'} at 127.0.0.1:35628
I20260809 19:34:05.362243 28718 tablet_replica.cc:330] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: stopping tablet replica
I20260809 19:34:05.362263 28721 tablet_replica.cc:330] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092: stopping tablet replica
I20260809 19:34:05.362353 28721 raft_consensus.cc:2229] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:05.362393 28718 raft_consensus.cc:2229] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:05.362509 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 28004
I20260809 19:34:05.362519 28721 raft_consensus.cc:2258] T e99d912a049e4c418d91e700d45888e2 P 787cece96584480c8a636a879d243092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:05.362565 28718 raft_consensus.cc:2258] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:05.369928 28720 tablet_replica.cc:330] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: stopping tablet replica
I20260809 19:34:05.370038 28720 raft_consensus.cc:2229] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:05.370136 28720 raft_consensus.cc:2258] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:05.370576 28718 ts_tablet_manager.cc:1905] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:05.372763 28718 ts_tablet_manager.cc:1918] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:05.372857 28718 log.cc:1196] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/e99d912a049e4c418d91e700d45888e2
I20260809 19:34:05.373232 28718 ts_tablet_manager.cc:1939] T e99d912a049e4c418d91e700d45888e2 P 26dc5e8eca7a4e61b3087e2286eb749a: Deleting consensus metadata
I20260809 19:34:05.373680 27874 catalog_manager.cc:4841] TS 26dc5e8eca7a4e61b3087e2286eb749a (127.31.57.194:41021): tablet e99d912a049e4c418d91e700d45888e2 (table test-table [id=a5e05059478143e99aeea23f912358ab]) successfully deleted
W20260809 19:34:05.375003 27874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:43743 (error 108)
I20260809 19:34:05.375286 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 28216
I20260809 19:34:05.376991 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:05.375411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:43743 (local address 127.0.0.1:35636)
0809 19:34:05.376912 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:05.376965 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1478}
W20260809 19:34:05.377139 27874 catalog_manager.cc:4584] TS 787cece96584480c8a636a879d243092 (127.31.57.193:43743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e99d912a049e4c418d91e700d45888e2: Network error: Client connection negotiation failed: client connection to 127.31.57.193:43743: connect: Connection refused (error 111)
I20260809 19:34:05.382104 28720 ts_tablet_manager.cc:1905] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:05.384500 28720 ts_tablet_manager.cc:1918] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:05.384573 28720 log.cc:1196] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/e99d912a049e4c418d91e700d45888e2
I20260809 19:34:05.384788 28720 ts_tablet_manager.cc:1939] T e99d912a049e4c418d91e700d45888e2 P 1b60ba9320fc448ba5639fa9a220d5dd: Deleting consensus metadata
I20260809 19:34:05.385800 27874 catalog_manager.cc:4841] TS 1b60ba9320fc448ba5639fa9a220d5dd (127.31.57.195:46015): tablet e99d912a049e4c418d91e700d45888e2 (table test-table [id=a5e05059478143e99aeea23f912358ab]) successfully deleted
I20260809 19:34:05.389469 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 28427
I20260809 19:34:05.400681 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 27849
2026-08-09T19:34:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (788 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:05 dist-test-slave-x4qh krb5kdc[28725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:05 dist-test-slave-x4qh krb5kdc[28725](info): set up 2 sockets
Aug 09 19:34:05 dist-test-slave-x4qh krb5kdc[28725](info): commencing operation
krb5kdc: starting...
W20260809 19:34:06.588858 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:06 dist-test-slave-x4qh krb5kdc[28725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:06.698105 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37781
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:37781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:06.774137 28751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:06.774317 28751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:06.774339 28751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:06.775480 28751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:06.775518 28751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:06.775532 28751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:06.775578 28751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:06.775596 28751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:06.776963 28751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:37781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:06.777118 28751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:06.777343 28751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:34:06.782071 28751 server_base.cc:1034] running on GCE node
W20260809 19:34:06.781999 28757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:06.782001 28767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:06.782007 28760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:06.782493 28751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:06.782768 28751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:06.783953 28751 hybrid_clock.cc:648] HybridClock initialized: now 1786304046783939 us; error 41 us; skew 500 ppm
Aug 09 19:34:06 dist-test-slave-x4qh krb5kdc[28725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304046, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:06.786334 28751 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:06.786727 28751 webserver.cc:460] Webserver started at http://127.31.57.254:42261/ using document root <none> and password file <none>
I20260809 19:34:06.786943 28751 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:06.786985 28751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:06.787062 28751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:06.788020 28751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "e7cf62364a6b4116bd03f695bea262a5"
format_stamp: "Formatted at 2026-08-09 19:34:06 on dist-test-slave-x4qh"
server_key: "3f439926fcc298820c723f6924e53762"
server_key_iv: "62d015fadc4cd31e442e80f2205d94c0"
server_key_version: "encryptionkey@0"
I20260809 19:34:06.788293 28751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "e7cf62364a6b4116bd03f695bea262a5"
format_stamp: "Formatted at 2026-08-09 19:34:06 on dist-test-slave-x4qh"
server_key: "3f439926fcc298820c723f6924e53762"
server_key_iv: "62d015fadc4cd31e442e80f2205d94c0"
server_key_version: "encryptionkey@0"
I20260809 19:34:06.789510 28751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:06.790134 28778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:06.790313 28751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:06.790390 28751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "e7cf62364a6b4116bd03f695bea262a5"
format_stamp: "Formatted at 2026-08-09 19:34:06 on dist-test-slave-x4qh"
server_key: "3f439926fcc298820c723f6924e53762"
server_key_iv: "62d015fadc4cd31e442e80f2205d94c0"
server_key_version: "encryptionkey@0"
I20260809 19:34:06.790439 28751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:06.837610 28751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:06.839321 28751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:06.839509 28751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:06.843559 28751 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:37781
I20260809 19:34:06.843623 28861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:37781 every 8 connection(s)
I20260809 19:34:06.843966 28751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:06.844815 28864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:06.847015 28864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: Bootstrap starting.
I20260809 19:34:06.847636 28864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:06.847951 28864 log.cc:824] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:06.848590 28864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: No bootstrap required, opened a new log
I20260809 19:34:06.850060 28864 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } }
I20260809 19:34:06.850199 28864 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:06.850260 28864 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7cf62364a6b4116bd03f695bea262a5, State: Initialized, Role: FOLLOWER
I20260809 19:34:06.850425 28864 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } }
I20260809 19:34:06.850519 28864 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:06.850559 28864 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:06.850605 28864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:06.851075 28864 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } }
I20260809 19:34:06.851184 28864 leader_election.cc:302] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7cf62364a6b4116bd03f695bea262a5; no voters:
I20260809 19:34:06.851359 28864 leader_election.cc:288] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:06.851430 28868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:06.851567 28868 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [term 1 LEADER]: Becoming Leader. State: Replica: e7cf62364a6b4116bd03f695bea262a5, State: Running, Role: LEADER
I20260809 19:34:06.851653 28864 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:06.851692 28868 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } }
I20260809 19:34:06.852285 28870 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7cf62364a6b4116bd03f695bea262a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } } }
I20260809 19:34:06.852384 28870 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:06.852537 28874 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cf62364a6b4116bd03f695bea262a5. Latest consensus state: current_term: 1 leader_uuid: "e7cf62364a6b4116bd03f695bea262a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cf62364a6b4116bd03f695bea262a5" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37781 } } }
I20260809 19:34:06.852665 28874 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:06.852723 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 28751
I20260809 19:34:06.852869 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:06.852854 28887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:06.853091 31975 external_mini_cluster.cc:1428] Setting key 1569b30cd6e8b2a8265815430ecf1d48
I20260809 19:34:06.853459 28887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:06.855046 28887 catalog_manager.cc:1347] Generated new cluster ID: 83c046c5bdf646d88352b9f2e64e8e6a
I20260809 19:34:06.855099 28887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:06 dist-test-slave-x4qh krb5kdc[28725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:06.864717 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:06.853514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.31.57.254:37781)
0809 19:34:06.853716 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:06.853724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:06.853944 (+ 220us) server_negotiation.cc:407] Connection header received
0809 19:34:06.854103 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:06.854110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:06.854178 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:06.854326 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:06.855103 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:06.856469 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:06.857486 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:06.858143 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:06.858264 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:06.861103 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:06.861107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:06.861107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:06.861119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:06.863071 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:06.863523 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:06.863526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:06.863526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:06.863637 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:06.863854 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:06.863854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:06.863855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:06.864113 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:06.864249 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:06.864395 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:34:06.864513 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260809 19:34:06.867898 28887 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:06.868333 28887 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:06.873463 28887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: Generated new TSK 0
I20260809 19:34:06.873642 28887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:34:06.902658 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:06.993950 28910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:06.994220 28910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:06.994258 28910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:06.995272 28910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:06.995334 28910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:06.995375 28910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:06.996646 28910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:06.996886 28910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:06.997083 28910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.001804 28930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.001835 28931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.001812 28934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.001950 28910 server_base.cc:1034] running on GCE node
I20260809 19:34:07.002184 28910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.002411 28910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.003544 28910 hybrid_clock.cc:648] HybridClock initialized: now 1786304047003535 us; error 40 us; skew 500 ppm
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:07.005952 28910 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:34:07.006309 28910 webserver.cc:460] Webserver started at http://127.31.57.193:42197/ using document root <none> and password file <none>
I20260809 19:34:07.006491 28910 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.006531 28910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.006616 28910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.007587 28910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "f916b10a0a654ebeb0f082d0010231b7"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "f78b6878e8d6a456f21cc1ca802f700e"
server_key_iv: "ebaf6e79f4131c08b91d0786622a28b1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.007889 28910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "f916b10a0a654ebeb0f082d0010231b7"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "f78b6878e8d6a456f21cc1ca802f700e"
server_key_iv: "ebaf6e79f4131c08b91d0786622a28b1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.009109 28910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:07.009748 28949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.009933 28910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:07.010007 28910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "f916b10a0a654ebeb0f082d0010231b7"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "f78b6878e8d6a456f21cc1ca802f700e"
server_key_iv: "ebaf6e79f4131c08b91d0786622a28b1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.010056 28910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.020027 28910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.021649 28910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.021751 28910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.021910 28910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:07.022136 28910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:07.022166 28910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.022186 28910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:07.022200 28910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.027441 28910 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:35813
I20260809 19:34:07.027485 29160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:35813 every 8 connection(s)
I20260809 19:34:07.027755 28910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:07.035488 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.028372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:59641 (local address 127.31.57.254:37781)
0809 19:34:07.028428 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.028430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.028622 (+ 192us) server_negotiation.cc:407] Connection header received
0809 19:34:07.029038 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.029039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.029047 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.029112 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.029902 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.030369 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.031218 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.031447 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.031496 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.033296 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.033298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.033298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:07.033304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.034379 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.034708 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.034709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.034709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.034753 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.034950 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.034951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.034951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.035039 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.035084 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.035292 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.035357 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:34:07.035542 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.028355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37781 (local address 127.31.57.193:59641)
0809 19:34:07.028508 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.028541 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.028931 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.029156 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.029159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.029636 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:07.029827 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.029832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.030459 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.030459 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.031146 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.031148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.031506 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.031507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.031596 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.031967 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:07.031974 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:07.033209 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:07.034448 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.034455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.034456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.034646 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.034811 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.034812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.034812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.034898 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.035091 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:07.035095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:07.035188 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.035259 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.035362 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260809 19:34:07.036034 29161 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37781
I20260809 19:34:07.036109 29161 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:07.036275 29161 heartbeater.cc:511] Master 127.31.57.254:37781 requested a full tablet report, sending...
I20260809 19:34:07.036754 28800 ts_manager.cc:194] Registered new tserver with Master: f916b10a0a654ebeb0f082d0010231b7 (127.31.57.193:35813)
I20260809 19:34:07.037279 28800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:59641
I20260809 19:34:07.037835 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 28910
I20260809 19:34:07.037958 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:07.038147 31975 external_mini_cluster.cc:1428] Setting key dda14252c2fc8e7cd836ebe0aa055a24
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:34:07.062304 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:07.136183 29167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.136335 29167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.136355 29167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.137362 29167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.137398 29167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.137422 29167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:07.138703 29167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.138870 29167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.139093 29167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.143721 29175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.143743 29178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.143761 29167 server_base.cc:1034] running on GCE node
W20260809 19:34:07.143743 29183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.144107 29167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.144331 29167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.145495 29167 hybrid_clock.cc:648] HybridClock initialized: now 1786304047145485 us; error 27 us; skew 500 ppm
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:07.147998 29167 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:34:07.148341 29167 webserver.cc:460] Webserver started at http://127.31.57.194:34195/ using document root <none> and password file <none>
I20260809 19:34:07.148517 29167 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.148553 29167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.148633 29167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.149510 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "de3d5e1f56e044da966175bf558ef980"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "7cba630ef02ede3be17e56051d23f867"
server_key_iv: "ddeba029eacce084de3335024a73930a"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.149821 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "de3d5e1f56e044da966175bf558ef980"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "7cba630ef02ede3be17e56051d23f867"
server_key_iv: "ddeba029eacce084de3335024a73930a"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.151021 29167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:07.151752 29195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.151953 29167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:34:07.152024 29167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "de3d5e1f56e044da966175bf558ef980"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "7cba630ef02ede3be17e56051d23f867"
server_key_iv: "ddeba029eacce084de3335024a73930a"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.152074 29167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.164582 29167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.166621 29167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.166769 29167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.166970 29167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:07.167259 29167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:07.167310 29167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.167348 29167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:07.167397 29167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.172938 29167 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:34573
I20260809 19:34:07.173009 29399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:34573 every 8 connection(s)
I20260809 19:34:07.173316 29167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:34:07.176553 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 29167
I20260809 19:34:07.176671 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:07.176862 31975 external_mini_cluster.cc:1428] Setting key 56904924da04f411cb547c2f3709d24d
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:07.182091 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.174024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:42931 (local address 127.31.57.254:37781)
0809 19:34:07.174073 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.174076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.174268 (+ 192us) server_negotiation.cc:407] Connection header received
0809 19:34:07.174657 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.174658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.174668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.174719 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.175584 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.176041 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.177088 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.177302 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.177353 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.179524 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.179525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.179525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:07.179530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.180669 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.181115 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.181116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.181116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.181162 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.181466 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.181466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.181467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.181552 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.181599 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.181965 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.182031 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:34:07.182417 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.174002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37781 (local address 127.31.57.194:42931)
0809 19:34:07.174161 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.174191 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.174580 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.174784 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.174787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.175271 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:07.175512 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.175517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.176114 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.176115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.177010 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.177012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.177461 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.177462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.177578 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.178009 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:07.178017 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:07.179443 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:07.180755 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.180762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.180764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.181044 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.181221 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.181222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.181223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.181373 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.181618 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:07.181625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:07.181778 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.181940 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.182114 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260809 19:34:07.182989 29400 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37781
I20260809 19:34:07.183092 29400 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:07.183300 29400 heartbeater.cc:511] Master 127.31.57.254:37781 requested a full tablet report, sending...
I20260809 19:34:07.183737 28800 ts_manager.cc:194] Registered new tserver with Master: de3d5e1f56e044da966175bf558ef980 (127.31.57.194:34573)
I20260809 19:34:07.184110 28800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:42931
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:34:07.203682 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:07.278412 29413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.278612 29413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.278644 29413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.279729 29413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.279790 29413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.279829 29413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:07.281183 29413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.281421 29413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.281617 29413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.286319 29423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.286342 29425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.286427 29413 server_base.cc:1034] running on GCE node
W20260809 19:34:07.286321 29428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.286744 29413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.286918 29413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.288043 29413 hybrid_clock.cc:648] HybridClock initialized: now 1786304047288031 us; error 27 us; skew 500 ppm
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:07.290066 29413 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:34:07.290424 29413 webserver.cc:460] Webserver started at http://127.31.57.195:33647/ using document root <none> and password file <none>
I20260809 19:34:07.290648 29413 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.290712 29413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.290856 29413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.291805 29413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "0e8f8abf7ad445bdb34413e7799f3569"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "459a6225e5a4a6a4b4853fb6061099af"
server_key_iv: "3de3a288164155a4b57cf1e9702f381c"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.292171 29413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "0e8f8abf7ad445bdb34413e7799f3569"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "459a6225e5a4a6a4b4853fb6061099af"
server_key_iv: "3de3a288164155a4b57cf1e9702f381c"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.293457 29413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:07.294207 29438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.294467 29413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:07.294552 29413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "0e8f8abf7ad445bdb34413e7799f3569"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "459a6225e5a4a6a4b4853fb6061099af"
server_key_iv: "3de3a288164155a4b57cf1e9702f381c"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.294631 29413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.319938 29413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.321681 29413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.321836 29413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.322086 29413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:07.322398 29413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:07.322454 29413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.322510 29413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:07.322537 29413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.328204 29413 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:39159
I20260809 19:34:07.328253 29629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:39159 every 8 connection(s)
I20260809 19:34:07.328580 29413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:34:07 dist-test-slave-x4qh krb5kdc[28725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304047, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:07.337172 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.329318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:51699 (local address 127.31.57.254:37781)
0809 19:34:07.329394 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.329396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.329551 (+ 155us) server_negotiation.cc:407] Connection header received
0809 19:34:07.330027 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.330029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.330039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.330095 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.331058 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.331592 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.332567 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.332740 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.332791 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.334881 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.334883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.334883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:07.334889 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.336071 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.336429 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.336430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.336430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.336477 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.336680 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.336681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.336681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.336774 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.336823 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.337044 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.337103 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:34:07.337311 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.329261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37781 (local address 127.31.57.195:51699)
0809 19:34:07.329441 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.329474 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.329921 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.330143 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.330146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.330654 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:07.330845 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.330850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.331692 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.331693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.332490 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.332493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.332814 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.332815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.332915 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.333441 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:07.333449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:07.334765 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:07.336143 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.336149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.336151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.336365 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.336536 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:07.336537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:07.336537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:07.336626 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:07.336829 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:07.336833 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:07.336935 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.337013 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.337123 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260809 19:34:07.337843 29633 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37781
I20260809 19:34:07.337949 29633 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:07.338116 29633 heartbeater.cc:511] Master 127.31.57.254:37781 requested a full tablet report, sending...
I20260809 19:34:07.338347 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 29413
I20260809 19:34:07.338464 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:07.338595 28800 ts_manager.cc:194] Registered new tserver with Master: 0e8f8abf7ad445bdb34413e7799f3569 (127.31.57.195:39159)
I20260809 19:34:07.338699 31975 external_mini_cluster.cc:1428] Setting key 6fb0480fcf8e8c8e9eaf159c2c3ab385
I20260809 19:34:07.339104 28800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:51699
I20260809 19:34:07.339716 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:07.346997 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.341134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.31.57.254:37781)
0809 19:34:07.341191 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.341193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.341323 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:34:07.341399 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.341400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.341411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.341460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.342018 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.342474 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.343213 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.343483 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.343554 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.344748 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.344749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.344749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:07.344756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.345818 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.346199 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.346200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.346200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.346251 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:07.346476 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:07.346477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:07.346478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:07.346603 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.346720 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.346845 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.346907 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:34:07.348662 28800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:07.350126 28800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:07.356631 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:34573 (local address 127.0.0.1:59124)
0809 19:34:07.352400 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.352436 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.352509 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.353153 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.353153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.353156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:07.353209 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.353210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354244 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.354245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.355648 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.355650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356301 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.356381 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356394 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.356489 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.356538 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260809 19:34:07.356743 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.31.57.195:39159)
0809 19:34:07.352508 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.352513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.353001 (+ 488us) server_negotiation.cc:407] Connection header received
0809 19:34:07.353052 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.353054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.353068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.353141 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:07.353338 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354379 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.355727 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356396 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356481 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356538 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.356625 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.356667 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260809 19:34:07.356928 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.31.57.193:35813)
0809 19:34:07.352679 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.352683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.352796 (+ 113us) server_negotiation.cc:407] Connection header received
0809 19:34:07.352846 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.352848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.352870 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.353351 (+ 481us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:07.353430 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354475 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.355774 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356431 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356490 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356538 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.356788 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.356839 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260809 19:34:07.356745 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:39159 (local address 127.0.0.1:33240)
0809 19:34:07.352705 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.352716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.352784 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.353231 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.353232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.353234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:07.353271 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.353273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354480 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.354481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.355646 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.355650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356489 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.356559 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356562 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.356615 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.356667 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":19,"thread_start_us":51,"threads_started":1}
I20260809 19:34:07.357025 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.31.57.194:34573)
0809 19:34:07.352920 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.352924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.352933 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:07.352988 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.352990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.353006 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.353293 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:07.353366 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354141 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.355744 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356204 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356300 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356356 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.356876 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.356929 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":110,"threads_started":1}
I20260809 19:34:07.356822 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.352310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:35813 (local address 127.0.0.1:42366)
0809 19:34:07.352705 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.352716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.352784 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.352979 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.352982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.352985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:07.353072 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.353074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.354600 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.354601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.355658 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.355662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.356575 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.356576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.356640 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.356644 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.356693 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.356730 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":48,"threads_started":1}
I20260809 19:34:07.357712 29030 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae (DEFAULT_TABLE table=test-table [id=f89ecd58fad7457287f21c2913835157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:07.357712 29278 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae (DEFAULT_TABLE table=test-table [id=f89ecd58fad7457287f21c2913835157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:07.357713 29513 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae (DEFAULT_TABLE table=test-table [id=f89ecd58fad7457287f21c2913835157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:07.358065 29030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6df937f75944b7b08fcca7411bc3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:07.358065 29278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6df937f75944b7b08fcca7411bc3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:07.358071 29513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6df937f75944b7b08fcca7411bc3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:07.360452 29655 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Bootstrap starting.
I20260809 19:34:07.361227 29657 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: Bootstrap starting.
I20260809 19:34:07.361280 29655 tablet_bootstrap.cc:654] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:07.361868 29655 log.cc:824] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:07.361989 29657 tablet_bootstrap.cc:654] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:07.362574 29657 log.cc:824] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:07.362660 29655 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: No bootstrap required, opened a new log
I20260809 19:34:07.362720 29655 ts_tablet_manager.cc:1397] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:07.362912 29656 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Bootstrap starting.
I20260809 19:34:07.363276 29657 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: No bootstrap required, opened a new log
I20260809 19:34:07.363363 29657 ts_tablet_manager.cc:1397] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:07.363587 29656 tablet_bootstrap.cc:654] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:07.364573 29656 log.cc:824] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:07.364784 29657 raft_consensus.cc:348] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.364982 29657 raft_consensus.cc:374] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:07.365063 29657 raft_consensus.cc:729] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f916b10a0a654ebeb0f082d0010231b7, State: Initialized, Role: FOLLOWER
I20260809 19:34:07.365103 29655 raft_consensus.cc:348] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.365207 29657 consensus_queue.cc:260] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.365273 29655 raft_consensus.cc:374] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:07.365336 29655 raft_consensus.cc:729] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8f8abf7ad445bdb34413e7799f3569, State: Initialized, Role: FOLLOWER
I20260809 19:34:07.365360 29656 tablet_bootstrap.cc:492] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: No bootstrap required, opened a new log
I20260809 19:34:07.365415 29656 ts_tablet_manager.cc:1397] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:34:07.365497 29657 ts_tablet_manager.cc:1428] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:07.365474 29655 consensus_queue.cc:260] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.365548 29161 heartbeater.cc:503] Master 127.31.57.254:37781 was elected leader, sending a full tablet report...
I20260809 19:34:07.365738 29655 ts_tablet_manager.cc:1428] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:07.365806 29633 heartbeater.cc:503] Master 127.31.57.254:37781 was elected leader, sending a full tablet report...
I20260809 19:34:07.367331 29656 raft_consensus.cc:348] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.367528 29656 raft_consensus.cc:374] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:07.367554 29656 raft_consensus.cc:729] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3d5e1f56e044da966175bf558ef980, State: Initialized, Role: FOLLOWER
I20260809 19:34:07.367678 29656 consensus_queue.cc:260] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.367933 29656 ts_tablet_manager.cc:1428] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:07.367973 29400 heartbeater.cc:503] Master 127.31.57.254:37781 was elected leader, sending a full tablet report...
I20260809 19:34:07.369961 29664 raft_consensus.cc:482] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:07.370033 29664 raft_consensus.cc:504] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.370361 29664 leader_election.cc:288] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e8f8abf7ad445bdb34413e7799f3569 (127.31.57.195:39159), f916b10a0a654ebeb0f082d0010231b7 (127.31.57.193:35813)
I20260809 19:34:07.372956 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.370523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:41987 (local address 127.31.57.195:39159)
0809 19:34:07.370560 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.370562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.370570 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:07.370622 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.370622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.370632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.370685 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:07.370868 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.371345 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372255 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.372608 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372686 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.372693 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.372869 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.372903 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:34:07.372956 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.370472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:39159 (local address 127.31.57.194:41987)
0809 19:34:07.370517 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.370527 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.370562 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.370728 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.370729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.370731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:07.370795 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.370796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.371468 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.371469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.372155 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.372157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.372698 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.372789 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.372792 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.372852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.372892 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:34:07.373027 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.370711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:50103 (local address 127.31.57.193:35813)
0809 19:34:07.370745 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.370747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.370755 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:07.370774 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.370775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.370782 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.370836 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:07.370973 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.371439 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372333 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.372702 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372784 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.372791 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.372937 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.372967 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:07.373066 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.370509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:35813 (local address 127.31.57.194:50103)
0809 19:34:07.370665 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.370673 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.370711 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.370856 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.370857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.370858 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:07.370914 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.370915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.371527 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.371529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.372242 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.372244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.372784 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.372785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.372855 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.372858 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.372931 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.372967 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260809 19:34:07.373383 29074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6df937f75944b7b08fcca7411bc3ae" candidate_uuid: "de3d5e1f56e044da966175bf558ef980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f916b10a0a654ebeb0f082d0010231b7" is_pre_election: true
I20260809 19:34:07.373459 29550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6df937f75944b7b08fcca7411bc3ae" candidate_uuid: "de3d5e1f56e044da966175bf558ef980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8f8abf7ad445bdb34413e7799f3569" is_pre_election: true
I20260809 19:34:07.373505 29074 raft_consensus.cc:2454] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de3d5e1f56e044da966175bf558ef980 in term 0.
I20260809 19:34:07.373570 29550 raft_consensus.cc:2454] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de3d5e1f56e044da966175bf558ef980 in term 0.
I20260809 19:34:07.373690 29198 leader_election.cc:302] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de3d5e1f56e044da966175bf558ef980, f916b10a0a654ebeb0f082d0010231b7; no voters:
I20260809 19:34:07.373852 29664 raft_consensus.cc:2781] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:07.373950 29664 raft_consensus.cc:482] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:07.373987 29664 raft_consensus.cc:3037] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:07.374562 29664 raft_consensus.cc:504] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.374702 29664 leader_election.cc:288] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [CANDIDATE]: Term 1 election: Requested vote from peers 0e8f8abf7ad445bdb34413e7799f3569 (127.31.57.195:39159), f916b10a0a654ebeb0f082d0010231b7 (127.31.57.193:35813)
I20260809 19:34:07.374862 29550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6df937f75944b7b08fcca7411bc3ae" candidate_uuid: "de3d5e1f56e044da966175bf558ef980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e8f8abf7ad445bdb34413e7799f3569"
I20260809 19:34:07.374881 29074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6df937f75944b7b08fcca7411bc3ae" candidate_uuid: "de3d5e1f56e044da966175bf558ef980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f916b10a0a654ebeb0f082d0010231b7"
I20260809 19:34:07.374938 29550 raft_consensus.cc:3037] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:07.374938 29074 raft_consensus.cc:3037] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:07.375751 29550 raft_consensus.cc:2454] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de3d5e1f56e044da966175bf558ef980 in term 1.
I20260809 19:34:07.375752 29074 raft_consensus.cc:2454] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de3d5e1f56e044da966175bf558ef980 in term 1.
I20260809 19:34:07.375933 29200 leader_election.cc:302] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8f8abf7ad445bdb34413e7799f3569, de3d5e1f56e044da966175bf558ef980; no voters:
I20260809 19:34:07.376041 29664 raft_consensus.cc:2781] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:07.376181 29664 raft_consensus.cc:686] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 1 LEADER]: Becoming Leader. State: Replica: de3d5e1f56e044da966175bf558ef980, State: Running, Role: LEADER
I20260809 19:34:07.376282 29664 consensus_queue.cc:237] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } }
I20260809 19:34:07.376917 28800 catalog_manager.cc:5495] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 reported cstate change: term changed from 0 to 1, leader changed from <none> to de3d5e1f56e044da966175bf558ef980 (127.31.57.194). New cstate: current_term: 1 leader_uuid: "de3d5e1f56e044da966175bf558ef980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de3d5e1f56e044da966175bf558ef980" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 34573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 } health_report { overall_health: UNKNOWN } } }
I20260809 19:34:07.383512 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.381433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.31.57.194:34573)
0809 19:34:07.381492 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.381494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.381501 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:07.381530 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.381531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.381552 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.381607 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:07.381711 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.382119 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.382823 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.382997 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.383046 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.383184 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:07.383259 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:07.383331 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.383430 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.383442 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:34:07.385008 29550 raft_consensus.cc:1261] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 1 FOLLOWER]: Refusing update from remote peer de3d5e1f56e044da966175bf558ef980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:07.385016 29074 raft_consensus.cc:1261] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 1 FOLLOWER]: Refusing update from remote peer de3d5e1f56e044da966175bf558ef980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:07.385360 29666 consensus_queue.cc:1035] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8f8abf7ad445bdb34413e7799f3569" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:07.385483 29664 consensus_queue.cc:1035] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f916b10a0a654ebeb0f082d0010231b7" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 35813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:07.387658 29672 mvcc.cc:205] Tried to move back new op lower bound from 7316701378086813696 to 7316701378053824512. Current Snapshot: MvccSnapshot[applied={T|T < 7316701378086813696}]
I20260809 19:34:07.387871 29674 mvcc.cc:205] Tried to move back new op lower bound from 7316701378086813696 to 7316701378053824512. Current Snapshot: MvccSnapshot[applied={T|T < 7316701378086813696}]
I20260809 19:34:07.388537 29675 mvcc.cc:205] Tried to move back new op lower bound from 7316701378086813696 to 7316701378053824512. Current Snapshot: MvccSnapshot[applied={T|T < 7316701378086813696}]
I20260809 19:34:07.390560 28800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:07.390667 28800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:07.391436 28800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7cf62364a6b4116bd03f695bea262a5: Sending DeleteTablet for 3 replicas of tablet 1c6df937f75944b7b08fcca7411bc3ae
I20260809 19:34:07.391786 29278 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:07 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:59124
I20260809 19:34:07.391780 29030 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:07 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:42366
I20260809 19:34:07.391997 29513 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6df937f75944b7b08fcca7411bc3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:07 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:33240
I20260809 19:34:07.392207 29701 tablet_replica.cc:330] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: stopping tablet replica
I20260809 19:34:07.392298 29700 tablet_replica.cc:330] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7: stopping tablet replica
I20260809 19:34:07.392361 29701 raft_consensus.cc:2229] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:07.392424 29700 raft_consensus.cc:2229] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:07.392488 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 28910
I20260809 19:34:07.392514 29701 raft_consensus.cc:2258] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:07.392551 29700 raft_consensus.cc:2258] T 1c6df937f75944b7b08fcca7411bc3ae P f916b10a0a654ebeb0f082d0010231b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:07.392994 29701 ts_tablet_manager.cc:1905] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:07.394685 29701 ts_tablet_manager.cc:1918] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:07.394744 29701 log.cc:1196] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/1c6df937f75944b7b08fcca7411bc3ae
I20260809 19:34:07.394973 29701 ts_tablet_manager.cc:1939] T 1c6df937f75944b7b08fcca7411bc3ae P 0e8f8abf7ad445bdb34413e7799f3569: Deleting consensus metadata
I20260809 19:34:07.409233 29697 tablet_replica.cc:330] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: stopping tablet replica
I20260809 19:34:07.409353 29697 raft_consensus.cc:2229] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:07.409469 29697 raft_consensus.cc:2258] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:07.410967 29697 ts_tablet_manager.cc:1905] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:07.412681 29697 ts_tablet_manager.cc:1918] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:07.412751 29697 log.cc:1196] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/1c6df937f75944b7b08fcca7411bc3ae
I20260809 19:34:07.412987 29697 ts_tablet_manager.cc:1939] T 1c6df937f75944b7b08fcca7411bc3ae P de3d5e1f56e044da966175bf558ef980: Deleting consensus metadata
I20260809 19:34:07.413470 28779 catalog_manager.cc:4841] TS de3d5e1f56e044da966175bf558ef980 (127.31.57.194:34573): tablet 1c6df937f75944b7b08fcca7411bc3ae (table test-table [id=f89ecd58fad7457287f21c2913835157]) successfully deleted
I20260809 19:34:07.420768 28784 catalog_manager.cc:4841] TS 0e8f8abf7ad445bdb34413e7799f3569 (127.31.57.195:39159): tablet 1c6df937f75944b7b08fcca7411bc3ae (table test-table [id=f89ecd58fad7457287f21c2913835157]) successfully deleted
W20260809 19:34:07.424968 28780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:35813 (error 108)
I20260809 19:34:07.425539 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.425377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:35813 (local address 127.0.0.1:42374)
0809 19:34:07.425441 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.425498 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:34:07.425623 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 29167
W20260809 19:34:07.425670 28780 catalog_manager.cc:4584] TS f916b10a0a654ebeb0f082d0010231b7 (127.31.57.193:35813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c6df937f75944b7b08fcca7411bc3ae: Network error: Client connection negotiation failed: client connection to 127.31.57.193:35813: connect: Connection refused (error 111)
I20260809 19:34:07.433827 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 29413
I20260809 19:34:07.442333 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 28751
2026-08-09T19:34:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2042 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T19:34:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:07Z Disabled control of system clock
I20260809 19:34:07.473834 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43543
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:07.553100 29708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.553295 29708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.553314 29708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.554409 29708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:07.554450 29708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.554463 29708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.554476 29708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:07.554524 29708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:07.555789 29708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:43543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.555989 29708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.556260 29708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.560788 29723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.560781 29726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.560796 29728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.560847 29708 server_base.cc:1034] running on GCE node
I20260809 19:34:07.561218 29708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.561501 29708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.562669 29708 hybrid_clock.cc:648] HybridClock initialized: now 1786304047562647 us; error 34 us; skew 500 ppm
I20260809 19:34:07.563850 29708 webserver.cc:460] Webserver started at http://127.31.57.254:37347/ using document root <none> and password file <none>
I20260809 19:34:07.564083 29708 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.564153 29708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.564278 29708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.565268 29708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "46d2bd6567d74e18b95d4874cf159d3c"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "a6ebf0fae55ad8185bf6f826a9159a9f"
server_key_iv: "fba09e6a685386e635a5620cfc72b8a1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.565603 29708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "46d2bd6567d74e18b95d4874cf159d3c"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "a6ebf0fae55ad8185bf6f826a9159a9f"
server_key_iv: "fba09e6a685386e635a5620cfc72b8a1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.566886 29708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:07.567633 29739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.567828 29708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:07.567912 29708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "46d2bd6567d74e18b95d4874cf159d3c"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "a6ebf0fae55ad8185bf6f826a9159a9f"
server_key_iv: "fba09e6a685386e635a5620cfc72b8a1"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.567961 29708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.589386 29708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.589632 29708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.589726 29708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.593524 29708 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:43543
I20260809 19:34:07.593578 29832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:43543 every 8 connection(s)
I20260809 19:34:07.593940 29708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:07.594910 29834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:07.597203 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: Bootstrap starting.
I20260809 19:34:07.597792 29834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:07.598097 29834 log.cc:824] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:07.598115 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 29708
I20260809 19:34:07.598227 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:07.598405 31975 external_mini_cluster.cc:1428] Setting key 8cc1dad0cf70f23271dcd20c833fb0b5
I20260809 19:34:07.598882 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: No bootstrap required, opened a new log
I20260809 19:34:07.601588 29834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } }
I20260809 19:34:07.601753 29834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:07.601792 29834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d2bd6567d74e18b95d4874cf159d3c, State: Initialized, Role: FOLLOWER
I20260809 19:34:07.601915 29834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } }
I20260809 19:34:07.601998 29834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:07.602035 29834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:07.602088 29834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:07.602614 29834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } }
I20260809 19:34:07.602751 29834 leader_election.cc:302] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46d2bd6567d74e18b95d4874cf159d3c; no voters:
I20260809 19:34:07.602955 29834 leader_election.cc:288] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:07.602984 29842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:07.603106 29842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [term 1 LEADER]: Becoming Leader. State: Replica: 46d2bd6567d74e18b95d4874cf159d3c, State: Running, Role: LEADER
I20260809 19:34:07.603259 29834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:07.603247 29842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } }
I20260809 19:34:07.603749 29845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d2bd6567d74e18b95d4874cf159d3c. Latest consensus state: current_term: 1 leader_uuid: "46d2bd6567d74e18b95d4874cf159d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } } }
I20260809 19:34:07.603838 29845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:07.604092 29855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:07.604058 29851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46d2bd6567d74e18b95d4874cf159d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2bd6567d74e18b95d4874cf159d3c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 43543 } } }
I20260809 19:34:07.604780 29851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:07.605140 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.598900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.31.57.254:43543)
0809 19:34:07.599113 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.599132 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.599168 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:34:07.600127 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.600136 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.600206 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.600361 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.600511 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.601926 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.602768 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.603360 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.603476 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.604384 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.604388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.604389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:07.604575 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.604606 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:07.604616 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.604692 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.604872 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.604951 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260809 19:34:07.605443 29855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:07.606834 29855 catalog_manager.cc:1347] Generated new cluster ID: bc08d4ab1d60423293bb6079a08e97e0
I20260809 19:34:07.606905 29855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:07.617069 29855 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:07.617782 29855 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:07.625247 29855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: Generated new TSK 0
I20260809 19:34:07.625408 29855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:07.626487 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--builtin_ntp_servers=127.31.57.212:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:07.706548 29879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.706722 29879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.706743 29879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.707921 29879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.707959 29879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.707983 29879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:07.709250 29879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.709472 29879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.709779 29879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.714669 29887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.714679 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.714658 29888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.715075 29879 server_base.cc:1034] running on GCE node
I20260809 19:34:07.715325 29879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.715618 29879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.716768 29879 hybrid_clock.cc:648] HybridClock initialized: now 1786304047716758 us; error 25 us; skew 500 ppm
I20260809 19:34:07.717962 29879 webserver.cc:460] Webserver started at http://127.31.57.193:46191/ using document root <none> and password file <none>
I20260809 19:34:07.718235 29879 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.718278 29879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.718416 29879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.719442 29879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "87c54082da824b4caa41868c6277caed"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "0e54f14a9ca8c3b014a22b7507b8630f"
server_key_iv: "272482ee57267d2f4d949692a04f3c86"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.719784 29879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "87c54082da824b4caa41868c6277caed"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "0e54f14a9ca8c3b014a22b7507b8630f"
server_key_iv: "272482ee57267d2f4d949692a04f3c86"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.721007 29879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:34:07.721858 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.722050 29879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:34:07.722139 29879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "87c54082da824b4caa41868c6277caed"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "0e54f14a9ca8c3b014a22b7507b8630f"
server_key_iv: "272482ee57267d2f4d949692a04f3c86"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.722226 29879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.734799 29879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.735057 29879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.735203 29879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.735481 29879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:07.735796 29879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:07.735852 29879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.735890 29879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:07.735916 29879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.741362 29879 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:36439
I20260809 19:34:07.741457 30087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:36439 every 8 connection(s)
I20260809 19:34:07.741745 29879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:34:07.746209 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.742468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:60079 (local address 127.31.57.254:43543)
0809 19:34:07.742522 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.742524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.742748 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:34:07.743172 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.743173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.743184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.743232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.743884 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.744322 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.745172 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.745336 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.745382 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.746020 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.746021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.746022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:07.746025 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.746037 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:07.746041 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.746078 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.746144 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.746162 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:34:07.746369 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.742435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43543 (local address 127.31.57.193:60079)
0809 19:34:07.742626 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.742662 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.743087 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.743285 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.743287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.743294 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:07.743816 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.743821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.744408 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.744409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.745101 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.745103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.745405 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.745406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.745503 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.745893 (+ 390us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:07.745893 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:07.745916 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:07.745936 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:07.746078 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:07.746080 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:07.746082 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.746120 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.746181 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260809 19:34:07.746877 30088 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43543
I20260809 19:34:07.746981 30088 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:07.747138 30088 heartbeater.cc:511] Master 127.31.57.254:43543 requested a full tablet report, sending...
I20260809 19:34:07.747617 29765 ts_manager.cc:194] Registered new tserver with Master: 87c54082da824b4caa41868c6277caed (127.31.57.193:36439)
I20260809 19:34:07.748214 29765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.193:60079
I20260809 19:34:07.751734 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 29879
I20260809 19:34:07.751849 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:07.752059 31975 external_mini_cluster.cc:1428] Setting key 247edb60b682e99a3e88015f2d924925
I20260809 19:34:07.752931 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--builtin_ntp_servers=127.31.57.212:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:07.831452 30092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.831640 30092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.831662 30092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.832719 30092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.832756 30092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.832782 30092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:07.834009 30092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.834197 30092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.834458 30092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.839126 30105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.839144 30102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.839145 30100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.839404 30092 server_base.cc:1034] running on GCE node
I20260809 19:34:07.839610 30092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.839831 30092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.840996 30092 hybrid_clock.cc:648] HybridClock initialized: now 1786304047840986 us; error 39 us; skew 500 ppm
I20260809 19:34:07.842229 30092 webserver.cc:460] Webserver started at http://127.31.57.194:37729/ using document root <none> and password file <none>
I20260809 19:34:07.842518 30092 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.842586 30092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.842741 30092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.843797 30092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "2c6cfbd250344474bbb09646540e6408"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "06800cc2873aeb6a5fd7a1b4f3f3c931"
server_key_iv: "cface124e7fe12fa368d440dd704d54e"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.844142 30092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "2c6cfbd250344474bbb09646540e6408"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "06800cc2873aeb6a5fd7a1b4f3f3c931"
server_key_iv: "cface124e7fe12fa368d440dd704d54e"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.845398 30092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:07.846056 30113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.846216 30092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:07.846318 30092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "2c6cfbd250344474bbb09646540e6408"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "06800cc2873aeb6a5fd7a1b4f3f3c931"
server_key_iv: "cface124e7fe12fa368d440dd704d54e"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.846395 30092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:07.879894 30092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:07.880192 30092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:07.880342 30092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:07.880556 30092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:07.880856 30092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:07.880905 30092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.880935 30092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:07.880961 30092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.887018 30092 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:44349
I20260809 19:34:07.887097 30290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:44349 every 8 connection(s)
I20260809 19:34:07.887485 30092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:34:07.888597 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 30092
I20260809 19:34:07.888698 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:07.888881 31975 external_mini_cluster.cc:1428] Setting key 2caa26e8ad10c14075fd8b9ed9d9e31b
I20260809 19:34:07.889814 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--builtin_ntp_servers=127.31.57.212:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:34:07.893290 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.888195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:50465 (local address 127.31.57.254:43543)
0809 19:34:07.888245 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:07.888247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:07.888437 (+ 190us) server_negotiation.cc:407] Connection header received
0809 19:34:07.889096 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:07.889098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:07.889110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:07.889170 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:07.889947 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.890612 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.891886 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.892105 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.892186 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.892982 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:07.892983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:07.892984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:07.892988 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:07.893009 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:07.893013 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:07.893074 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:07.893195 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 19:34:07.893222 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:07.893481 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:07.888142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43543 (local address 127.31.57.194:50465)
0809 19:34:07.888316 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:07.888351 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:07.889000 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:07.889232 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:07.889235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:07.889244 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:07.889834 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.889841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.890739 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.890740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.891801 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:07.891804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:07.892194 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:07.892195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:07.892334 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:07.892850 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:07.892850 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:07.892875 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:07.892897 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:07.893074 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:07.893081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:07.893084 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:07.893197 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 19:34:07.893284 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260809 19:34:07.893966 30291 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43543
I20260809 19:34:07.894049 30291 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:07.894202 30291 heartbeater.cc:511] Master 127.31.57.254:43543 requested a full tablet report, sending...
I20260809 19:34:07.894562 29765 ts_manager.cc:194] Registered new tserver with Master: 2c6cfbd250344474bbb09646540e6408 (127.31.57.194:44349)
I20260809 19:34:07.894914 29765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.194:50465
W20260809 19:34:07.966607 30294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:07.966781 30294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:07.966802 30294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:07.967912 30294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:07.967949 30294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:07.967970 30294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:07.969116 30294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:43543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:07.969327 30294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:07.969579 30294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:07.974117 30306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.974226 30294 server_base.cc:1034] running on GCE node
W20260809 19:34:07.974224 30311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:07.974128 30304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:07.974560 30294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:07.974786 30294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:07.975940 30294 hybrid_clock.cc:648] HybridClock initialized: now 1786304047975918 us; error 51 us; skew 500 ppm
I20260809 19:34:07.977043 30294 webserver.cc:460] Webserver started at http://127.31.57.195:46663/ using document root <none> and password file <none>
I20260809 19:34:07.977262 30294 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:07.977327 30294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:07.977476 30294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:07.978369 30294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "41565b4340514166943de9ca364265fa"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "648d07f9b2f9089127453724523401b1"
server_key_iv: "a19fc4455391a165b4c2317bddc94d1b"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.978686 30294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "41565b4340514166943de9ca364265fa"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "648d07f9b2f9089127453724523401b1"
server_key_iv: "a19fc4455391a165b4c2317bddc94d1b"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.979897 30294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:07.980510 30319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:07.980656 30294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:07.980762 30294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "41565b4340514166943de9ca364265fa"
format_stamp: "Formatted at 2026-08-09 19:34:07 on dist-test-slave-x4qh"
server_key: "648d07f9b2f9089127453724523401b1"
server_key_iv: "a19fc4455391a165b4c2317bddc94d1b"
server_key_version: "encryptionkey@0"
I20260809 19:34:07.980842 30294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:08.001907 30294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:08.002169 30294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:08.002319 30294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:08.002525 30294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:08.002822 30294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:08.002877 30294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:08.002915 30294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:08.002941 30294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:08.008420 30294 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:42397
I20260809 19:34:08.008492 30511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:42397 every 8 connection(s)
I20260809 19:34:08.008781 30294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:34:08.013702 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.009433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:36293 (local address 127.31.57.254:43543)
0809 19:34:08.009500 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.009501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.009722 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:34:08.010204 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.010206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.010218 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.010270 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:08.010877 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.011341 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.012309 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.012522 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.012599 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.013420 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:08.013422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:08.013422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:08.013427 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:08.013442 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:08.013446 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:08.013493 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.013591 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.013617 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:34:08.013899 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.009427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:43543 (local address 127.31.57.195:36293)
0809 19:34:08.009608 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.009636 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.010070 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.010332 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.010336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.010344 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:08.010785 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.010790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.011475 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.011476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.012211 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.012214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.012610 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.012611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.012747 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.013273 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:08.013274 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:08.013299 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:08.013321 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:08.013504 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:08.013509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:08.013513 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.013591 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.013687 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260809 19:34:08.014346 30512 heartbeater.cc:347] Connected to a master server at 127.31.57.254:43543
I20260809 19:34:08.014421 30512 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:08.014613 30512 heartbeater.cc:511] Master 127.31.57.254:43543 requested a full tablet report, sending...
I20260809 19:34:08.014961 29765 ts_manager.cc:194] Registered new tserver with Master: 41565b4340514166943de9ca364265fa (127.31.57.195:42397)
I20260809 19:34:08.015298 29765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.195:36293
I20260809 19:34:08.015517 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 30294
I20260809 19:34:08.015614 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:08.015796 31975 external_mini_cluster.cc:1428] Setting key 4ea72dd398d322bb0d6f1d0e781e2b9b
I20260809 19:34:08.016537 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:08.020393 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.017909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.31.57.254:43543)
0809 19:34:08.017977 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.017979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.018104 (+ 125us) server_negotiation.cc:407] Connection header received
0809 19:34:08.018140 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.018140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.018149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.018195 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:08.018332 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.018759 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.019468 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.019621 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.019699 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.020201 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:08.020202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:08.020202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:08.020206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:08.020216 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:08.020219 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:08.020282 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.020341 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.020349 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:34:08.021785 29765 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:08.023290 29765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:08.028934 30531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:42397 (local address 127.0.0.1:43856)
0809 19:34:08.025750 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.025768 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.025828 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.026157 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.026157 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.026159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:08.026195 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.026196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.027221 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.027222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.027987 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.027990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.028679 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.028680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.028746 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.028765 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.028827 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.028869 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":73,"threads_started":1}
I20260809 19:34:08.028957 30536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.31.57.195:42397)
0809 19:34:08.025721 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.025725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.025821 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:34:08.025883 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.025885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.025897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.025974 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:08.026274 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.027136 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.028057 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.028594 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.028674 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.028739 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.028857 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.028889 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":65,"threads_started":1}
I20260809 19:34:08.029487 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:44349 (local address 127.0.0.1:51220)
0809 19:34:08.025494 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.025539 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.025603 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.025881 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.025884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.025886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:08.025950 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.025952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.027062 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.027063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.028295 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.028298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.029274 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.029274 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.029332 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.029335 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.029394 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.029430 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":42,"thread_start_us":59,"threads_started":1}
I20260809 19:34:08.029522 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.31.57.194:44349)
0809 19:34:08.025730 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.025735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.025744 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:08.025786 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.025787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.025801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.025872 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:08.026033 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.026958 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.028384 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.029199 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.029272 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.029347 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.029407 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.029447 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260809 19:34:08.029662 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.31.57.193:36439)
0809 19:34:08.025787 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.025791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.025963 (+ 172us) server_negotiation.cc:407] Connection header received
0809 19:34:08.026009 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.026011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.026024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.026086 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:08.026410 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.027037 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.028394 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.029073 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.029143 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.029221 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.029531 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.029571 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260809 19:34:08.029678 30396 tablet_service.cc:1467] Processing CreateTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 (DEFAULT_TABLE table=test-table [id=1c280f2ced074276b9085b941436733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:08.029819 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.025377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36439 (local address 127.0.0.1:43354)
0809 19:34:08.025907 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.025915 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.025957 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.026310 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.026310 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.026312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:08.026340 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.026341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.027124 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.027125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.028301 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.028302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.029321 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.029322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.029384 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.029388 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.029439 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.029474 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260809 19:34:08.030012 30396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02455ffc3aa2444db4cf1c17f1bfc260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:08.030210 30189 tablet_service.cc:1467] Processing CreateTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 (DEFAULT_TABLE table=test-table [id=1c280f2ced074276b9085b941436733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:08.030478 30189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02455ffc3aa2444db4cf1c17f1bfc260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:08.032621 30541 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Bootstrap starting.
I20260809 19:34:08.032936 30542 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Bootstrap starting.
I20260809 19:34:08.033285 30541 tablet_bootstrap.cc:654] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:08.033327 29980 tablet_service.cc:1467] Processing CreateTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 (DEFAULT_TABLE table=test-table [id=1c280f2ced074276b9085b941436733c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:08.033536 30542 tablet_bootstrap.cc:654] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:08.033612 30541 log.cc:824] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:08.033653 29980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02455ffc3aa2444db4cf1c17f1bfc260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:08.033854 30542 log.cc:824] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:08.034510 30542 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: No bootstrap required, opened a new log
I20260809 19:34:08.034588 30542 ts_tablet_manager.cc:1397] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:08.036104 30541 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: No bootstrap required, opened a new log
I20260809 19:34:08.036190 30541 ts_tablet_manager.cc:1397] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 19:34:08.036494 30548 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Bootstrap starting.
I20260809 19:34:08.037073 30548 tablet_bootstrap.cc:654] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:08.036975 30542 raft_consensus.cc:348] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.037129 30542 raft_consensus.cc:374] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:08.037160 30542 raft_consensus.cc:729] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c6cfbd250344474bbb09646540e6408, State: Initialized, Role: FOLLOWER
I20260809 19:34:08.037292 30542 consensus_queue.cc:260] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.037369 30548 log.cc:824] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:08.037535 30542 ts_tablet_manager.cc:1428] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:08.037676 30291 heartbeater.cc:503] Master 127.31.57.254:43543 was elected leader, sending a full tablet report...
I20260809 19:34:08.038097 30548 tablet_bootstrap.cc:492] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: No bootstrap required, opened a new log
I20260809 19:34:08.038184 30548 ts_tablet_manager.cc:1397] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:08.038642 30541 raft_consensus.cc:348] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.038825 30541 raft_consensus.cc:374] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:08.038864 30541 raft_consensus.cc:729] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41565b4340514166943de9ca364265fa, State: Initialized, Role: FOLLOWER
I20260809 19:34:08.038976 30541 consensus_queue.cc:260] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.039202 30541 ts_tablet_manager.cc:1428] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:08.039247 30512 heartbeater.cc:503] Master 127.31.57.254:43543 was elected leader, sending a full tablet report...
I20260809 19:34:08.040448 30548 raft_consensus.cc:348] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.040549 30548 raft_consensus.cc:374] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:08.040566 30548 raft_consensus.cc:729] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c54082da824b4caa41868c6277caed, State: Initialized, Role: FOLLOWER
I20260809 19:34:08.040674 30548 consensus_queue.cc:260] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.040928 30548 ts_tablet_manager.cc:1428] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:08.040979 30088 heartbeater.cc:503] Master 127.31.57.254:43543 was elected leader, sending a full tablet report...
W20260809 19:34:08.138123 30292 tablet.cc:2369] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:34:08.242440 30090 tablet.cc:2369] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:08.248756 30553 raft_consensus.cc:482] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:08.248857 30553 raft_consensus.cc:504] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.249195 30553 leader_election.cc:288] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c6cfbd250344474bbb09646540e6408 (127.31.57.194:44349), 87c54082da824b4caa41868c6277caed (127.31.57.193:36439)
I20260809 19:34:08.251984 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.249391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:44349 (local address 127.31.57.195:54301)
0809 19:34:08.249439 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.249450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.249521 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.249667 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.249668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.249669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:08.249740 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.249741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.250400 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.250401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.251161 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.251163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.251736 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.251821 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.251825 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.251878 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.251907 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":3}
I20260809 19:34:08.252018 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.249506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:54301 (local address 127.31.57.194:44349)
0809 19:34:08.249552 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.249554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.249562 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:08.249591 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.249592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.249599 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.249664 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:08.249816 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.250320 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251246 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.251660 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251714 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.251721 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.251887 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.251929 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:34:08.252117 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.249392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36439 (local address 127.31.57.195:46145)
0809 19:34:08.249559 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.249568 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:08.249620 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:08.249762 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:08.249763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:08.249764 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:08.249829 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.249831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.250520 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.250521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.251264 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:08.251266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.251821 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:08.251911 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.251915 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:08.251975 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:34:08.252025 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":13,"thread_start_us":88,"threads_started":1}
I20260809 19:34:08.252133 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.249552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:46145 (local address 127.31.57.193:36439)
0809 19:34:08.249593 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.249594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.249605 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:34:08.249670 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.249671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.249680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.249740 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:08.249893 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.250449 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251353 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.251741 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.251809 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.251817 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.251990 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.252052 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:34:08.252401 30009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02455ffc3aa2444db4cf1c17f1bfc260" candidate_uuid: "41565b4340514166943de9ca364265fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c54082da824b4caa41868c6277caed" is_pre_election: true
I20260809 19:34:08.252415 30223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02455ffc3aa2444db4cf1c17f1bfc260" candidate_uuid: "41565b4340514166943de9ca364265fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c6cfbd250344474bbb09646540e6408" is_pre_election: true
I20260809 19:34:08.252547 30009 raft_consensus.cc:2454] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41565b4340514166943de9ca364265fa in term 0.
I20260809 19:34:08.252552 30223 raft_consensus.cc:2454] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41565b4340514166943de9ca364265fa in term 0.
I20260809 19:34:08.252787 30322 leader_election.cc:302] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c6cfbd250344474bbb09646540e6408, 41565b4340514166943de9ca364265fa; no voters:
I20260809 19:34:08.252921 30553 raft_consensus.cc:2781] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:08.252977 30553 raft_consensus.cc:482] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:08.253000 30553 raft_consensus.cc:3037] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:08.253647 30553 raft_consensus.cc:504] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.253779 30553 leader_election.cc:288] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [CANDIDATE]: Term 1 election: Requested vote from peers 2c6cfbd250344474bbb09646540e6408 (127.31.57.194:44349), 87c54082da824b4caa41868c6277caed (127.31.57.193:36439)
I20260809 19:34:08.253921 30009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02455ffc3aa2444db4cf1c17f1bfc260" candidate_uuid: "41565b4340514166943de9ca364265fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c54082da824b4caa41868c6277caed"
I20260809 19:34:08.253921 30223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02455ffc3aa2444db4cf1c17f1bfc260" candidate_uuid: "41565b4340514166943de9ca364265fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c6cfbd250344474bbb09646540e6408"
I20260809 19:34:08.254001 30223 raft_consensus.cc:3037] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:08.254004 30009 raft_consensus.cc:3037] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:08.254612 30223 raft_consensus.cc:2454] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41565b4340514166943de9ca364265fa in term 1.
I20260809 19:34:08.254746 30009 raft_consensus.cc:2454] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41565b4340514166943de9ca364265fa in term 1.
I20260809 19:34:08.254761 30322 leader_election.cc:302] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c6cfbd250344474bbb09646540e6408, 41565b4340514166943de9ca364265fa; no voters:
I20260809 19:34:08.254917 30553 raft_consensus.cc:2781] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:08.255051 30553 raft_consensus.cc:686] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 1 LEADER]: Becoming Leader. State: Replica: 41565b4340514166943de9ca364265fa, State: Running, Role: LEADER
I20260809 19:34:08.255137 30553 consensus_queue.cc:237] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } }
I20260809 19:34:08.255930 29765 catalog_manager.cc:5495] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 41565b4340514166943de9ca364265fa (127.31.57.195). New cstate: current_term: 1 leader_uuid: "41565b4340514166943de9ca364265fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41565b4340514166943de9ca364265fa" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 42397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 } health_report { overall_health: UNKNOWN } } }
W20260809 19:34:08.259229 30519 tablet.cc:2369] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:08.310443 30536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.308068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.31.57.195:42397)
0809 19:34:08.308108 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:08.308110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:08.308118 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:08.308202 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:08.308203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:08.308229 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:08.308310 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:08.308448 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.308891 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.309636 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:08.309829 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:08.309906 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:08.310056 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:08.310144 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:08.310260 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:08.310318 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 19:34:08.310326 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:08.312285 30009 raft_consensus.cc:1261] T 02455ffc3aa2444db4cf1c17f1bfc260 P 87c54082da824b4caa41868c6277caed [term 1 FOLLOWER]: Refusing update from remote peer 41565b4340514166943de9ca364265fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:08.312609 30223 raft_consensus.cc:1261] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 1 FOLLOWER]: Refusing update from remote peer 41565b4340514166943de9ca364265fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:08.312639 30559 consensus_queue.cc:1035] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c54082da824b4caa41868c6277caed" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 36439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:08.312922 30559 consensus_queue.cc:1035] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c6cfbd250344474bbb09646540e6408" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:08.315048 30570 mvcc.cc:205] Tried to move back new op lower bound from 7316701381884313600 to 7316701381653549056. Current Snapshot: MvccSnapshot[applied={T|T < 7316701381884313600}]
I20260809 19:34:08.316390 30573 mvcc.cc:205] Tried to move back new op lower bound from 7316701381884313600 to 7316701381653549056. Current Snapshot: MvccSnapshot[applied={T|T < 7316701381884313600}]
I20260809 19:34:08.317042 29765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:08.317132 29765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:08.317840 29765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46d2bd6567d74e18b95d4874cf159d3c: Sending DeleteTablet for 3 replicas of tablet 02455ffc3aa2444db4cf1c17f1bfc260
I20260809 19:34:08.318167 30189 tablet_service.cc:1514] Processing DeleteTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:08 UTC) from {username='slave'} at 127.0.0.1:51220
I20260809 19:34:08.318195 29980 tablet_service.cc:1514] Processing DeleteTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:08 UTC) from {username='slave'} at 127.0.0.1:43354
I20260809 19:34:08.318282 30396 tablet_service.cc:1514] Processing DeleteTablet for tablet 02455ffc3aa2444db4cf1c17f1bfc260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:08 UTC) from {username='slave'} at 127.0.0.1:43856
I20260809 19:34:08.318588 30585 tablet_replica.cc:330] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: stopping tablet replica
I20260809 19:34:08.318634 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 29879
I20260809 19:34:08.318677 30585 raft_consensus.cc:2229] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:08.326215 30585 raft_consensus.cc:2258] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:08.327495 30585 ts_tablet_manager.cc:1905] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:08.329239 30585 ts_tablet_manager.cc:1918] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:08.329312 30585 log.cc:1196] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/02455ffc3aa2444db4cf1c17f1bfc260
I20260809 19:34:08.329572 30585 ts_tablet_manager.cc:1939] T 02455ffc3aa2444db4cf1c17f1bfc260 P 41565b4340514166943de9ca364265fa: Deleting consensus metadata
I20260809 19:34:08.329957 30584 tablet_replica.cc:330] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: stopping tablet replica
I20260809 19:34:08.330057 30584 raft_consensus.cc:2229] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:08.330160 30584 raft_consensus.cc:2258] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:08.335546 30584 ts_tablet_manager.cc:1905] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:08.337767 30584 ts_tablet_manager.cc:1918] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:08.337863 30584 log.cc:1196] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/02455ffc3aa2444db4cf1c17f1bfc260
I20260809 19:34:08.338169 30584 ts_tablet_manager.cc:1939] T 02455ffc3aa2444db4cf1c17f1bfc260 P 2c6cfbd250344474bbb09646540e6408: Deleting consensus metadata
I20260809 19:34:08.340269 29740 catalog_manager.cc:4841] TS 41565b4340514166943de9ca364265fa (127.31.57.195:42397): tablet 02455ffc3aa2444db4cf1c17f1bfc260 (table test-table [id=1c280f2ced074276b9085b941436733c]) successfully deleted
I20260809 19:34:08.340296 29741 catalog_manager.cc:4841] TS 2c6cfbd250344474bbb09646540e6408 (127.31.57.194:44349): tablet 02455ffc3aa2444db4cf1c17f1bfc260 (table test-table [id=1c280f2ced074276b9085b941436733c]) successfully deleted
W20260809 19:34:08.344980 29744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:36439 (error 108)
I20260809 19:34:08.345335 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 30092
I20260809 19:34:08.350957 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:08.350817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:36439 (local address 127.0.0.1:43356)
0809 19:34:08.350866 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:08.350915 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:34:08.351511 29744 catalog_manager.cc:4584] TS 87c54082da824b4caa41868c6277caed (127.31.57.193:36439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02455ffc3aa2444db4cf1c17f1bfc260: Network error: Client connection negotiation failed: client connection to 127.31.57.193:36439: connect: Connection refused (error 111)
I20260809 19:34:08.355597 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 30294
I20260809 19:34:08.364928 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 29708
2026-08-09T19:34:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:08 dist-test-slave-x4qh krb5kdc[30592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:08 dist-test-slave-x4qh krb5kdc[30592](info): set up 2 sockets
Aug 09 19:34:08 dist-test-slave-x4qh krb5kdc[30592](info): commencing operation
krb5kdc: starting...
W20260809 19:34:09.550056 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:09 dist-test-slave-x4qh krb5kdc[30592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:09.658149 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:36477
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:36477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:09.734045 30616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:09.734216 30616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:09.734236 30616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:09.735205 30616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:09.735236 30616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:09.735249 30616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:09.735260 30616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:09.735272 30616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:09.736613 30616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:36477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:36477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:09.736790 30616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:09.736948 30616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:09.741197 30626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:09.741205 30630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:09.741235 30616 server_base.cc:1034] running on GCE node
W20260809 19:34:09.741217 30625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:09.741643 30616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:09.741935 30616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:09.743093 30616 hybrid_clock.cc:648] HybridClock initialized: now 1786304049743072 us; error 44 us; skew 500 ppm
Aug 09 19:34:09 dist-test-slave-x4qh krb5kdc[30592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304049, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:09.752493 30616 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:09.752972 30616 webserver.cc:460] Webserver started at http://127.31.57.254:40773/ using document root <none> and password file <none>
I20260809 19:34:09.753218 30616 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:09.753297 30616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:09.753430 30616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:09.754379 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "6a7548d8291a4d739b74bd7744b13f74"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "0cae75b72d17e9d814aa993a18745bf0"
server_key_iv: "a2e224e9f5466cb5687bf3f347a3e8e7"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.754729 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "6a7548d8291a4d739b74bd7744b13f74"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "0cae75b72d17e9d814aa993a18745bf0"
server_key_iv: "a2e224e9f5466cb5687bf3f347a3e8e7"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.756106 30616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:34:09.756807 30646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:09.756978 30616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:09.757085 30616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "6a7548d8291a4d739b74bd7744b13f74"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "0cae75b72d17e9d814aa993a18745bf0"
server_key_iv: "a2e224e9f5466cb5687bf3f347a3e8e7"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.757161 30616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:09.779651 30616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:09.781242 30616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:09.781395 30616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:09.785394 30616 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:36477
I20260809 19:34:09.785429 30732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:36477 every 8 connection(s)
I20260809 19:34:09.785724 30616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:09.786499 30733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:09.788542 30733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: Bootstrap starting.
I20260809 19:34:09.789072 30733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:09.789354 30733 log.cc:824] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:09.790055 30733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: No bootstrap required, opened a new log
I20260809 19:34:09.791608 30733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } }
I20260809 19:34:09.791725 30733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:09.791745 30733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7548d8291a4d739b74bd7744b13f74, State: Initialized, Role: FOLLOWER
I20260809 19:34:09.791900 30733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } }
I20260809 19:34:09.791957 30733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:09.791996 30733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:09.792061 30733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:09.792620 30733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } }
I20260809 19:34:09.792690 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 30616
I20260809 19:34:09.792734 30733 leader_election.cc:302] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a7548d8291a4d739b74bd7744b13f74; no voters:
I20260809 19:34:09.792809 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:09.792906 30733 leader_election.cc:288] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:09.792935 30737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:09.793012 31975 external_mini_cluster.cc:1428] Setting key 26845f9d073dc3f23e80b310325e71da
I20260809 19:34:09.793089 30737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [term 1 LEADER]: Becoming Leader. State: Replica: 6a7548d8291a4d739b74bd7744b13f74, State: Running, Role: LEADER
I20260809 19:34:09.793152 30733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:09.793202 30737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } }
I20260809 19:34:09.793522 30737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a7548d8291a4d739b74bd7744b13f74. Latest consensus state: current_term: 1 leader_uuid: "6a7548d8291a4d739b74bd7744b13f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } } }
I20260809 19:34:09.793607 30737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:09.793758 30739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a7548d8291a4d739b74bd7744b13f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a7548d8291a4d739b74bd7744b13f74" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 36477 } } }
I20260809 19:34:09.793839 30757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:09.793861 30739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:09.794453 30757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:09.796169 30757 catalog_manager.cc:1347] Generated new cluster ID: f71a5d6ccc78424dab1e6df7d5530ebd
I20260809 19:34:09.796221 30757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:09 dist-test-slave-x4qh krb5kdc[30592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:09.803072 30757 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:09.803701 30757 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:09.805042 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:09.793574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.31.57.254:36477)
0809 19:34:09.794543 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:09.794552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:09.794582 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:34:09.794697 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:09.794704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:09.794768 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:09.794905 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:09.795692 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.796988 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.798103 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.798724 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.798811 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:09.801370 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:09.801374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:09.801374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:09.801387 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:09.803394 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:09.803775 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:09.803778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:09.803778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:09.803901 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:09.804123 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:09.804124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:09.804124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:09.804456 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:09.804619 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:09.804690 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 19:34:09.804832 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":60,"threads_started":1}
I20260809 19:34:09.809001 30757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: Generated new TSK 0
I20260809 19:34:09.809158 30757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.57.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
Entry for principal kudu/127.31.57.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab.
I20260809 19:34:09.834154 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:09.911779 30780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:09.911955 30780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:09.911976 30780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:09.913069 30780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:09.913110 30780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:09.913134 30780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:09.914469 30780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.193.keytab
--principal=kudu/127.31.57.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:09.914714 30780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:09.914932 30780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:09.919864 30791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:09.919853 30793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:09.919935 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:09.920164 30780 server_base.cc:1034] running on GCE node
I20260809 19:34:09.920359 30780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:09.920547 30780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:09.921785 30780 hybrid_clock.cc:648] HybridClock initialized: now 1786304049921770 us; error 31 us; skew 500 ppm
Aug 09 19:34:09 dist-test-slave-x4qh krb5kdc[30592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304049, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:09.924166 30780 init.cc:377] Logged in from keytab as kudu/127.31.57.193@KRBTEST.COM (short username kudu)
I20260809 19:34:09.924633 30780 webserver.cc:460] Webserver started at http://127.31.57.193:43625/ using document root <none> and password file <none>
I20260809 19:34:09.924854 30780 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:09.924921 30780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:09.925069 30780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:09.926015 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "450bd14193c34460aef43c5b37c48b4f"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "72c26f729e25a4d508ea10be1d25c92e"
server_key_iv: "d26aa8511a5a3a528fee8654e0529467"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.926370 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "450bd14193c34460aef43c5b37c48b4f"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "72c26f729e25a4d508ea10be1d25c92e"
server_key_iv: "d26aa8511a5a3a528fee8654e0529467"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.927668 30780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:09.928431 30807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:09.928604 30780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:09.928700 30780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "450bd14193c34460aef43c5b37c48b4f"
format_stamp: "Formatted at 2026-08-09 19:34:09 on dist-test-slave-x4qh"
server_key: "72c26f729e25a4d508ea10be1d25c92e"
server_key_iv: "d26aa8511a5a3a528fee8654e0529467"
server_key_version: "encryptionkey@0"
I20260809 19:34:09.928785 30780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:09.939316 30780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:09.940943 30780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:09.941105 30780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:09.941321 30780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:09.941612 30780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:09.941671 30780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:09.941705 30780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:09.941730 30780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:09.947546 30780 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:33755
I20260809 19:34:09.947590 30987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:33755 every 8 connection(s)
I20260809 19:34:09.947852 30780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:34:09.948385 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 30780
I20260809 19:34:09.948508 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:09.948686 31975 external_mini_cluster.cc:1428] Setting key 58e84558b40f8eff22c03a94370fe304
Aug 09 19:34:09 dist-test-slave-x4qh krb5kdc[30592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304049, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.193@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.57.194@KRBTEST.COM; defaulting to no policy
I20260809 19:34:09.957612 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:09.948468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:55903 (local address 127.31.57.254:36477)
0809 19:34:09.948536 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:09.948538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:09.948752 (+ 214us) server_negotiation.cc:407] Connection header received
0809 19:34:09.949257 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:09.949258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:09.949271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:09.949321 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:09.950138 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.950626 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.952188 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.952397 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.952464 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:09.954872 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:09.954873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:09.954874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:09.954881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:09.956275 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:09.956681 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:09.956682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:09.956683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:09.956743 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:09.957004 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:09.957005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:09.957005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:09.957111 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:09.957179 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:09.957447 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 19:34:09.957530 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:34:09.957762 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:09.948395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36477 (local address 127.31.57.193:55903)
0809 19:34:09.948611 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:09.948653 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:09.949146 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:09.949394 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:09.949398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:09.949887 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:09.950045 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:09.950049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.950725 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.950726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:09.952073 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:09.952077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:09.952496 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:09.952498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:09.952646 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:09.953134 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:09.953144 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:09.954774 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:09.956376 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:09.956384 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:09.956385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:09.956598 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:09.956822 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:09.956823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:09.956823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:09.956936 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:09.957171 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:09.957178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:09.957327 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 19:34:09.957426 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:34:09.957563 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260809 19:34:09.958233 30988 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36477
I20260809 19:34:09.958326 30988 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:09.958455 30988 heartbeater.cc:511] Master 127.31.57.254:36477 requested a full tablet report, sending...
I20260809 19:34:09.958992 30670 ts_manager.cc:194] Registered new tserver with Master: 450bd14193c34460aef43c5b37c48b4f (127.31.57.193:33755)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.194@KRBTEST.COM" created.
I20260809 19:34:09.959666 30670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.193@KRBTEST.COM'} at 127.31.57.193:55903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
Entry for principal kudu/127.31.57.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab.
I20260809 19:34:09.972173 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:10.048049 31000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.048213 31000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.048234 31000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.049166 31000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.049198 31000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.049222 31000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:10.050526 31000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.194.keytab
--principal=kudu/127.31.57.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.050698 31000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.050841 31000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.055472 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.055444 31014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.055456 31007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.055699 31000 server_base.cc:1034] running on GCE node
I20260809 19:34:10.055884 31000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:10.056130 31000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:10.057287 31000 hybrid_clock.cc:648] HybridClock initialized: now 1786304050057271 us; error 32 us; skew 500 ppm
Aug 09 19:34:10 dist-test-slave-x4qh krb5kdc[30592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:10.059545 31000 init.cc:377] Logged in from keytab as kudu/127.31.57.194@KRBTEST.COM (short username kudu)
I20260809 19:34:10.060084 31000 webserver.cc:460] Webserver started at http://127.31.57.194:46103/ using document root <none> and password file <none>
I20260809 19:34:10.060346 31000 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:10.060415 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:10.060541 31000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:10.061432 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "6e5571c115194cb1a9e59155b9286d9c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "024811a53404967936b88a47c8564cdb"
server_key_iv: "fab6b74b879cb41bfabb35ef5a094f5a"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.061786 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "6e5571c115194cb1a9e59155b9286d9c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "024811a53404967936b88a47c8564cdb"
server_key_iv: "fab6b74b879cb41bfabb35ef5a094f5a"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.063131 31000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:10.063915 31030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.064075 31000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.064167 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "6e5571c115194cb1a9e59155b9286d9c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "024811a53404967936b88a47c8564cdb"
server_key_iv: "fab6b74b879cb41bfabb35ef5a094f5a"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.064246 31000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:10.076480 31000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:10.078081 31000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:10.078230 31000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:10.078455 31000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:10.078742 31000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:10.078795 31000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.078830 31000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:10.078855 31000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.084308 31000 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:41707
I20260809 19:34:10.084349 31211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:41707 every 8 connection(s)
I20260809 19:34:10.084645 31000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:34:10.086376 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 31000
I20260809 19:34:10.086450 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:10.086611 31975 external_mini_cluster.cc:1428] Setting key 28623b8f1e2ebc531c92a06de27c66f1
Aug 09 19:34:10 dist-test-slave-x4qh krb5kdc[30592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.194@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:10.094509 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.085304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:49265 (local address 127.31.57.254:36477)
0809 19:34:10.085389 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.085391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.085541 (+ 150us) server_negotiation.cc:407] Connection header received
0809 19:34:10.085984 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.085985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.085998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.086055 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.087110 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.087811 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.088851 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.089119 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.089181 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.091550 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.091552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.091552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:10.091560 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.093040 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.093475 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.093476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.093477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.093557 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.093875 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.093876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.093877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.093987 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.094062 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.094340 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.094421 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 19:34:10.094671 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.085232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36477 (local address 127.31.57.194:49265)
0809 19:34:10.085418 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.085455 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.085883 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.086114 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.086118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.086715 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:10.087023 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.087028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.087910 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.087910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.088749 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.088752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.089241 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.089242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.089394 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.089838 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:10.089848 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:10.091456 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:10.093126 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:10.093133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:10.093135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:10.093386 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:10.093671 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:10.093672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:10.093673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:10.093776 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:10.094066 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:10.094073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:10.094197 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.094315 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.094459 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260809 19:34:10.095187 31212 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36477
I20260809 19:34:10.095271 31212 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:10.095463 31212 heartbeater.cc:511] Master 127.31.57.254:36477 requested a full tablet report, sending...
I20260809 19:34:10.095846 30670 ts_manager.cc:194] Registered new tserver with Master: 6e5571c115194cb1a9e59155b9286d9c (127.31.57.194:41707)
I20260809 19:34:10.096207 30670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.194@KRBTEST.COM'} at 127.31.57.194:49265
WARNING: no policy specified for kudu/127.31.57.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
Entry for principal kudu/127.31.57.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab.
I20260809 19:34:10.112351 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:10.187352 31221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.187537 31221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.187561 31221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.188617 31221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.188654 31221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.188679 31221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:10.190277 31221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.195.keytab
--principal=kudu/127.31.57.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:36477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.190490 31221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.190652 31221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.195631 31231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.195626 31237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.195652 31228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.195622 31221 server_base.cc:1034] running on GCE node
I20260809 19:34:10.195955 31221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:10.196139 31221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:10.197304 31221 hybrid_clock.cc:648] HybridClock initialized: now 1786304050197293 us; error 28 us; skew 500 ppm
Aug 09 19:34:10 dist-test-slave-x4qh krb5kdc[30592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:10.199703 31221 init.cc:377] Logged in from keytab as kudu/127.31.57.195@KRBTEST.COM (short username kudu)
I20260809 19:34:10.200138 31221 webserver.cc:460] Webserver started at http://127.31.57.195:45473/ using document root <none> and password file <none>
I20260809 19:34:10.200309 31221 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:10.200345 31221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:10.200438 31221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:10.201292 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "579a791d805d4c358b4ede9d2f1fe520"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "7f5730fef3528baf9adfe7651fa6fe3d"
server_key_iv: "9db0fbf4f8c6e47e0d421f7afadcd3af"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.201557 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "579a791d805d4c358b4ede9d2f1fe520"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "7f5730fef3528baf9adfe7651fa6fe3d"
server_key_iv: "9db0fbf4f8c6e47e0d421f7afadcd3af"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.202765 31221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:10.203464 31250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.203639 31221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.203739 31221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "579a791d805d4c358b4ede9d2f1fe520"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "7f5730fef3528baf9adfe7651fa6fe3d"
server_key_iv: "9db0fbf4f8c6e47e0d421f7afadcd3af"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.203835 31221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:10.225831 31221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:10.227547 31221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:10.227694 31221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:10.227905 31221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:10.228204 31221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:10.228255 31221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.228284 31221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:10.228309 31221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.234018 31221 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:40109
I20260809 19:34:10.234052 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:40109 every 8 connection(s)
I20260809 19:34:10.234349 31221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:34:10.236867 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 31221
I20260809 19:34:10.236979 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:10.237152 31975 external_mini_cluster.cc:1428] Setting key 557d1ad4d978a185b0f5cd4f358cd417
Aug 09 19:34:10 dist-test-slave-x4qh krb5kdc[30592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304050, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.195@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:10.243520 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.235062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:36205 (local address 127.31.57.254:36477)
0809 19:34:10.235134 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.235148 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.235322 (+ 174us) server_negotiation.cc:407] Connection header received
0809 19:34:10.235784 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.235785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.235794 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.235839 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.236707 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.237456 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.238693 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.238916 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.238979 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.241030 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.241031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.241031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:10.241038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.242147 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.242549 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.242550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.242550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.242613 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.242886 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.242887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.242888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.242997 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.243072 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.243327 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.243423 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:34:10.243686 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.235043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:36477 (local address 127.31.57.195:36205)
0809 19:34:10.235207 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.235242 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.235702 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.235891 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.235894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.236422 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:10.236590 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.236594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.237547 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.237548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.238509 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.238511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.239008 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.239009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.239121 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.239665 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:10.239672 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:10.240933 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:10.242250 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:10.242257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:10.242259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:10.242470 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:10.242687 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:10.242688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:10.242688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:10.242811 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:10.243063 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:10.243070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:10.243197 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.243304 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.243487 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260809 19:34:10.244059 31458 heartbeater.cc:347] Connected to a master server at 127.31.57.254:36477
I20260809 19:34:10.244128 31458 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:10.244315 31458 heartbeater.cc:511] Master 127.31.57.254:36477 requested a full tablet report, sending...
I20260809 19:34:10.244671 30670 ts_manager.cc:194] Registered new tserver with Master: 579a791d805d4c358b4ede9d2f1fe520 (127.31.57.195:40109)
I20260809 19:34:10.245019 30670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.57.195@KRBTEST.COM'} at 127.31.57.195:36205
I20260809 19:34:10.248623 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:10.255007 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.249871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36970 (local address 127.31.57.254:36477)
0809 19:34:10.249907 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.249909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.249999 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:34:10.250046 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.250046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.250053 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.250095 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.250598 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.250989 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.251681 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.251841 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.251889 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.252988 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.252988 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.252989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:10.252994 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.254071 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.254356 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.254357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.254357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.254399 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:10.254566 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:10.254566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:10.254566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:10.254692 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.254763 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.254849 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.254901 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:10.256384 30670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:10.257922 30670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:10.264194 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.259903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:41707 (local address 127.0.0.1:51824)
0809 19:34:10.260122 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.260131 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.260184 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.260637 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.260638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.260640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:10.260683 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.260684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261959 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.261959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.262952 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.262955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.263899 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.263900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.263981 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.263996 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.264073 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.264122 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260809 19:34:10.264328 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.260131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.31.57.195:40109)
0809 19:34:10.260330 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.260334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.260347 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:34:10.260385 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.260386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.260400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.260487 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:10.260681 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261800 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.263216 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.263916 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.264038 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.264104 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.264202 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.264235 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260809 19:34:10.264519 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.260139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.31.57.194:41707)
0809 19:34:10.260337 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.260343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.260354 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:34:10.260395 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.260397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.260412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.260490 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:10.260763 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261838 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.263048 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.263805 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.263885 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.263946 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.264396 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.264441 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260809 19:34:10.264590 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.260391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53808 (local address 127.31.57.193:33755)
0809 19:34:10.260642 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.260646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.260656 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:34:10.260695 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.260697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.260712 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.260798 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:10.261034 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261716 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.263449 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.263984 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.264071 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.264125 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.264450 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.264501 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":99,"threads_started":1}
I20260809 19:34:10.264598 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.259867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40109 (local address 127.0.0.1:34994)
0809 19:34:10.259968 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.260000 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.260080 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.260490 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.260493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.260494 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:10.260555 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.260557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261888 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.261889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.263131 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.263134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.264046 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.264047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.264126 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.264130 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.264188 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.264231 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":107,"thread_start_us":59,"threads_started":1}
I20260809 19:34:10.265154 31107 tablet_service.cc:1467] Processing CreateTablet for tablet 69f52d0085404f38a037c9cdeb936a2b (DEFAULT_TABLE table=test-table [id=f43296b6f6f34d11add61cf2922a21dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:10.265211 31330 tablet_service.cc:1467] Processing CreateTablet for tablet 69f52d0085404f38a037c9cdeb936a2b (DEFAULT_TABLE table=test-table [id=f43296b6f6f34d11add61cf2922a21dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:10.265520 31330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f52d0085404f38a037c9cdeb936a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:10.265520 31107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f52d0085404f38a037c9cdeb936a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:10.264724 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.259903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33755 (local address 127.0.0.1:53808)
0809 19:34:10.260330 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.260340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.260387 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.260882 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.260883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.260885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:10.260935 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.260936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.261821 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.261822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.263130 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.263134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.264159 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.264160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.264217 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.264221 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.264272 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.264664 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":67,"thread_start_us":64,"threads_started":1}
I20260809 19:34:10.268129 30882 tablet_service.cc:1467] Processing CreateTablet for tablet 69f52d0085404f38a037c9cdeb936a2b (DEFAULT_TABLE table=test-table [id=f43296b6f6f34d11add61cf2922a21dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:10.268504 30882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f52d0085404f38a037c9cdeb936a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:10.268923 31475 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Bootstrap starting.
I20260809 19:34:10.269601 31475 tablet_bootstrap.cc:654] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:10.270200 31475 log.cc:824] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:10.270973 31475 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: No bootstrap required, opened a new log
I20260809 19:34:10.271049 31475 ts_tablet_manager.cc:1397] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:10.271713 31477 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Bootstrap starting.
I20260809 19:34:10.272065 31481 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Bootstrap starting.
I20260809 19:34:10.272457 31477 tablet_bootstrap.cc:654] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:10.272709 31481 tablet_bootstrap.cc:654] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:10.273021 31477 log.cc:824] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:10.273109 31475 raft_consensus.cc:348] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.273273 31481 log.cc:824] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:10.273316 31475 raft_consensus.cc:374] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:10.273366 31475 raft_consensus.cc:729] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5571c115194cb1a9e59155b9286d9c, State: Initialized, Role: FOLLOWER
I20260809 19:34:10.273506 31475 consensus_queue.cc:260] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.273804 31475 ts_tablet_manager.cc:1428] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:10.273836 31212 heartbeater.cc:503] Master 127.31.57.254:36477 was elected leader, sending a full tablet report...
I20260809 19:34:10.273926 31477 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: No bootstrap required, opened a new log
I20260809 19:34:10.273974 31477 ts_tablet_manager.cc:1397] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:34:10.275772 31481 tablet_bootstrap.cc:492] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: No bootstrap required, opened a new log
I20260809 19:34:10.275830 31481 ts_tablet_manager.cc:1397] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 19:34:10.275879 31477 raft_consensus.cc:348] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.276041 31477 raft_consensus.cc:374] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:10.276085 31477 raft_consensus.cc:729] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579a791d805d4c358b4ede9d2f1fe520, State: Initialized, Role: FOLLOWER
I20260809 19:34:10.276189 31477 consensus_queue.cc:260] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.276444 31477 ts_tablet_manager.cc:1428] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:34:10.276501 31458 heartbeater.cc:503] Master 127.31.57.254:36477 was elected leader, sending a full tablet report...
I20260809 19:34:10.277745 31481 raft_consensus.cc:348] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.277869 31481 raft_consensus.cc:374] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:10.277890 31481 raft_consensus.cc:729] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 450bd14193c34460aef43c5b37c48b4f, State: Initialized, Role: FOLLOWER
I20260809 19:34:10.277997 31481 consensus_queue.cc:260] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.278203 31481 ts_tablet_manager.cc:1428] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:34:10.278277 30988 heartbeater.cc:503] Master 127.31.57.254:36477 was elected leader, sending a full tablet report...
W20260809 19:34:10.335232 31213 tablet.cc:2369] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:10.409628 31486 raft_consensus.cc:482] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:10.409741 31486 raft_consensus.cc:504] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.410073 31486 leader_election.cc:288] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579a791d805d4c358b4ede9d2f1fe520 (127.31.57.195:40109), 450bd14193c34460aef43c5b37c48b4f (127.31.57.193:33755)
I20260809 19:34:10.413996 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.410333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:53655 (local address 127.31.57.195:40109)
0809 19:34:10.410379 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.410381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.410390 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:10.410421 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.410422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.410432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.410512 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:10.410737 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.411485 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413033 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.413622 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.413707 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.413887 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.413927 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:34:10.413996 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.410362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:40225 (local address 127.31.57.193:33755)
0809 19:34:10.410426 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.410429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.410568 (+ 139us) server_negotiation.cc:407] Connection header received
0809 19:34:10.410604 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.410605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.410612 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.410655 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:10.410824 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.411496 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413033 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.413622 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.413707 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.413887 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.413927 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:34:10.414004 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.410237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40109 (local address 127.31.57.194:53655)
0809 19:34:10.410294 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.410305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.410346 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.410542 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.410543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.410545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:10.410620 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.410622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.411575 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.411575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.412929 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.412932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.413715 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.413806 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.413808 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.413859 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.413914 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":109}
I20260809 19:34:10.414011 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.410266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33755 (local address 127.31.57.194:40225)
0809 19:34:10.410444 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.410452 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.410549 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.410673 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.410674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.410676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:10.410755 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.410756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.411574 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.411575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.412929 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.412932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.413715 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.413716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.413794 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.413798 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.413857 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.413914 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":59,"thread_start_us":102,"threads_started":1}
I20260809 19:34:10.414422 30919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f52d0085404f38a037c9cdeb936a2b" candidate_uuid: "6e5571c115194cb1a9e59155b9286d9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "450bd14193c34460aef43c5b37c48b4f" is_pre_election: true
I20260809 19:34:10.414489 31359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f52d0085404f38a037c9cdeb936a2b" candidate_uuid: "6e5571c115194cb1a9e59155b9286d9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579a791d805d4c358b4ede9d2f1fe520" is_pre_election: true
I20260809 19:34:10.414580 30919 raft_consensus.cc:2454] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5571c115194cb1a9e59155b9286d9c in term 0.
I20260809 19:34:10.414597 31359 raft_consensus.cc:2454] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5571c115194cb1a9e59155b9286d9c in term 0.
I20260809 19:34:10.414795 31036 leader_election.cc:302] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 450bd14193c34460aef43c5b37c48b4f, 6e5571c115194cb1a9e59155b9286d9c; no voters:
I20260809 19:34:10.414918 31486 raft_consensus.cc:2781] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:10.414971 31486 raft_consensus.cc:482] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:10.414992 31486 raft_consensus.cc:3037] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:10.415705 31486 raft_consensus.cc:504] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.415851 31486 leader_election.cc:288] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [CANDIDATE]: Term 1 election: Requested vote from peers 579a791d805d4c358b4ede9d2f1fe520 (127.31.57.195:40109), 450bd14193c34460aef43c5b37c48b4f (127.31.57.193:33755)
I20260809 19:34:10.416014 30919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f52d0085404f38a037c9cdeb936a2b" candidate_uuid: "6e5571c115194cb1a9e59155b9286d9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "450bd14193c34460aef43c5b37c48b4f"
I20260809 19:34:10.416054 31359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f52d0085404f38a037c9cdeb936a2b" candidate_uuid: "6e5571c115194cb1a9e59155b9286d9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579a791d805d4c358b4ede9d2f1fe520"
I20260809 19:34:10.416108 30919 raft_consensus.cc:3037] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:10.416119 31359 raft_consensus.cc:3037] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:10.417033 30919 raft_consensus.cc:2454] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5571c115194cb1a9e59155b9286d9c in term 1.
I20260809 19:34:10.417033 31359 raft_consensus.cc:2454] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5571c115194cb1a9e59155b9286d9c in term 1.
I20260809 19:34:10.417210 31036 leader_election.cc:302] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 450bd14193c34460aef43c5b37c48b4f, 6e5571c115194cb1a9e59155b9286d9c; no voters:
I20260809 19:34:10.417344 31486 raft_consensus.cc:2781] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:10.417502 31486 raft_consensus.cc:686] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 1 LEADER]: Becoming Leader. State: Replica: 6e5571c115194cb1a9e59155b9286d9c, State: Running, Role: LEADER
I20260809 19:34:10.417595 31486 consensus_queue.cc:237] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } }
I20260809 19:34:10.418470 30670 catalog_manager.cc:5495] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5571c115194cb1a9e59155b9286d9c (127.31.57.194). New cstate: current_term: 1 leader_uuid: "6e5571c115194cb1a9e59155b9286d9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5571c115194cb1a9e59155b9286d9c" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 41707 } health_report { overall_health: HEALTHY } } }
I20260809 19:34:10.447402 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.445045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.31.57.194:41707)
0809 19:34:10.445105 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.445106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.445116 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:34:10.445168 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.445169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.445192 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.445263 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:10.445421 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.445858 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.446592 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.446813 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.446885 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.447041 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:10.447133 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:10.447221 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.447240 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.447257 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 19:34:10.448493 30990 tablet.cc:2369] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:34:10.449084 30919 raft_consensus.cc:1261] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 1 FOLLOWER]: Refusing update from remote peer 6e5571c115194cb1a9e59155b9286d9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:10.449095 31359 raft_consensus.cc:1261] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 1 FOLLOWER]: Refusing update from remote peer 6e5571c115194cb1a9e59155b9286d9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:10.449440 31486 consensus_queue.cc:1035] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "450bd14193c34460aef43c5b37c48b4f" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:10.449579 31494 consensus_queue.cc:1035] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "579a791d805d4c358b4ede9d2f1fe520" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:10.451853 31499 mvcc.cc:205] Tried to move back new op lower bound from 7316701390636748800 to 7316701390511251456. Current Snapshot: MvccSnapshot[applied={T|T < 7316701390636748800}]
I20260809 19:34:10.452111 31501 mvcc.cc:205] Tried to move back new op lower bound from 7316701390636748800 to 7316701390511251456. Current Snapshot: MvccSnapshot[applied={T|T < 7316701390636748800}]
I20260809 19:34:10.452471 31502 mvcc.cc:205] Tried to move back new op lower bound from 7316701390636748800 to 7316701390511251456. Current Snapshot: MvccSnapshot[applied={T|T < 7316701390636748800}]
I20260809 19:34:10.454017 30670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:10.454123 30670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:10.454873 30670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a7548d8291a4d739b74bd7744b13f74: Sending DeleteTablet for 3 replicas of tablet 69f52d0085404f38a037c9cdeb936a2b
I20260809 19:34:10.455215 31330 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f52d0085404f38a037c9cdeb936a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:10 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:34994
I20260809 19:34:10.455199 31107 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f52d0085404f38a037c9cdeb936a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:10 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:51824
I20260809 19:34:10.455190 30882 tablet_service.cc:1514] Processing DeleteTablet for tablet 69f52d0085404f38a037c9cdeb936a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:10 UTC) from {username='kudu', principal='kudu/127.31.57.254@KRBTEST.COM'} at 127.0.0.1:53808
I20260809 19:34:10.455497 31517 tablet_replica.cc:330] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f: stopping tablet replica
I20260809 19:34:10.455531 31513 tablet_replica.cc:330] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: stopping tablet replica
I20260809 19:34:10.455600 31517 raft_consensus.cc:2229] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:10.455622 31513 raft_consensus.cc:2229] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:10.455698 31517 raft_consensus.cc:2258] T 69f52d0085404f38a037c9cdeb936a2b P 450bd14193c34460aef43c5b37c48b4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:10.455715 31513 raft_consensus.cc:2258] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:10.455813 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 30780
I20260809 19:34:10.456108 31513 ts_tablet_manager.cc:1905] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:10.457904 31513 ts_tablet_manager.cc:1918] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:10.457964 31513 log.cc:1196] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/69f52d0085404f38a037c9cdeb936a2b
I20260809 19:34:10.458195 31513 ts_tablet_manager.cc:1939] T 69f52d0085404f38a037c9cdeb936a2b P 579a791d805d4c358b4ede9d2f1fe520: Deleting consensus metadata
I20260809 19:34:10.458631 30648 catalog_manager.cc:4841] TS 579a791d805d4c358b4ede9d2f1fe520 (127.31.57.195:40109): tablet 69f52d0085404f38a037c9cdeb936a2b (table test-table [id=f43296b6f6f34d11add61cf2922a21dc]) successfully deleted
I20260809 19:34:10.475095 31515 tablet_replica.cc:330] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: stopping tablet replica
I20260809 19:34:10.475215 31515 raft_consensus.cc:2229] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:10.475314 31515 raft_consensus.cc:2258] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:10.475917 31515 ts_tablet_manager.cc:1905] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:10.477217 31515 ts_tablet_manager.cc:1918] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:10.477317 31515 log.cc:1196] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/69f52d0085404f38a037c9cdeb936a2b
I20260809 19:34:10.477598 31515 ts_tablet_manager.cc:1939] T 69f52d0085404f38a037c9cdeb936a2b P 6e5571c115194cb1a9e59155b9286d9c: Deleting consensus metadata
I20260809 19:34:10.478018 30649 catalog_manager.cc:4841] TS 6e5571c115194cb1a9e59155b9286d9c (127.31.57.194:41707): tablet 69f52d0085404f38a037c9cdeb936a2b (table test-table [id=f43296b6f6f34d11add61cf2922a21dc]) successfully deleted
W20260809 19:34:10.481101 30650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:33755 (error 108)
I20260809 19:34:10.481248 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 31000
I20260809 19:34:10.481600 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.481491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:33755 (local address 127.0.0.1:37128)
0809 19:34:10.481534 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.481562 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:34:10.481694 30650 catalog_manager.cc:4584] TS 450bd14193c34460aef43c5b37c48b4f (127.31.57.193:33755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69f52d0085404f38a037c9cdeb936a2b: Network error: Client connection negotiation failed: client connection to 127.31.57.193:33755: connect: Connection refused (error 111)
I20260809 19:34:10.490851 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 31221
I20260809 19:34:10.497687 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 30616
2026-08-09T19:34:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2132 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T19:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:10Z Disabled control of system clock
I20260809 19:34:10.528872 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37865
--webserver_interface=127.31.57.254
--webserver_port=0
--builtin_ntp_servers=127.31.57.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:37865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:10.605808 31523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.606006 31523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.606025 31523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.607100 31523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:10.607138 31523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.607149 31523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.607161 31523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:10.607205 31523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:10.608433 31523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:37865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:37865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.608670 31523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.608943 31523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.613488 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.613541 31538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.613497 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.613731 31523 server_base.cc:1034] running on GCE node
I20260809 19:34:10.614023 31523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:10.614332 31523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:10.615429 31523 hybrid_clock.cc:648] HybridClock initialized: now 1786304050615404 us; error 37 us; skew 500 ppm
I20260809 19:34:10.616672 31523 webserver.cc:460] Webserver started at http://127.31.57.254:43063/ using document root <none> and password file <none>
I20260809 19:34:10.616925 31523 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:10.616979 31523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:10.617148 31523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:10.618142 31523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "c705c04a1e714c3e807f34cac3c49e5c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "68aa7795f3bfc92c6d011fed8eb5416f"
server_key_iv: "8139c7d86dc9f15ab6fd71c7aca5a461"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.618520 31523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "c705c04a1e714c3e807f34cac3c49e5c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "68aa7795f3bfc92c6d011fed8eb5416f"
server_key_iv: "8139c7d86dc9f15ab6fd71c7aca5a461"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.619803 31523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:34:10.620481 31546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.620692 31523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:10.620795 31523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "c705c04a1e714c3e807f34cac3c49e5c"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "68aa7795f3bfc92c6d011fed8eb5416f"
server_key_iv: "8139c7d86dc9f15ab6fd71c7aca5a461"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.620877 31523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:10.628911 31523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:10.629168 31523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:10.629331 31523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:10.633435 31523 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:37865
I20260809 19:34:10.633466 31647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:37865 every 8 connection(s)
I20260809 19:34:10.633795 31523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:10.634693 31648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:10.636812 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: Bootstrap starting.
I20260809 19:34:10.637279 31648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:10.637494 31648 log.cc:824] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:10.638087 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: No bootstrap required, opened a new log
I20260809 19:34:10.639647 31648 raft_consensus.cc:348] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } }
I20260809 19:34:10.639770 31648 raft_consensus.cc:374] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:10.639806 31648 raft_consensus.cc:729] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c705c04a1e714c3e807f34cac3c49e5c, State: Initialized, Role: FOLLOWER
I20260809 19:34:10.639901 31648 consensus_queue.cc:260] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } }
I20260809 19:34:10.639976 31648 raft_consensus.cc:388] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:10.640013 31648 raft_consensus.cc:482] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:10.640080 31648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:10.640583 31648 raft_consensus.cc:504] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } }
I20260809 19:34:10.640693 31648 leader_election.cc:302] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c705c04a1e714c3e807f34cac3c49e5c; no voters:
I20260809 19:34:10.640868 31648 leader_election.cc:288] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:10.640918 31653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:10.641088 31653 raft_consensus.cc:686] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [term 1 LEADER]: Becoming Leader. State: Replica: c705c04a1e714c3e807f34cac3c49e5c, State: Running, Role: LEADER
I20260809 19:34:10.641099 31648 sys_catalog.cc:564] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:10.641224 31653 consensus_queue.cc:237] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } }
I20260809 19:34:10.641724 31654 sys_catalog.cc:455] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c705c04a1e714c3e807f34cac3c49e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } } }
I20260809 19:34:10.641811 31654 sys_catalog.cc:458] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:10.641978 31655 sys_catalog.cc:455] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c705c04a1e714c3e807f34cac3c49e5c. Latest consensus state: current_term: 1 leader_uuid: "c705c04a1e714c3e807f34cac3c49e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c705c04a1e714c3e807f34cac3c49e5c" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 37865 } } }
I20260809 19:34:10.642055 31655 sys_catalog.cc:458] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:10.642197 31663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:10.642616 31663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:10.643173 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 31523
I20260809 19:34:10.643289 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:10.643491 31975 external_mini_cluster.cc:1428] Setting key 42805dbfd995e306472b35c7a49f6b45
I20260809 19:34:10.647862 31663 catalog_manager.cc:1347] Generated new cluster ID: 85e7bba52ec449de82a90163fc074e27
I20260809 19:34:10.647918 31663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:10.648795 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.643908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.31.57.254:37865)
0809 19:34:10.644106 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.644114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.644167 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:34:10.644280 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.644286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.644351 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.644481 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.644704 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.645651 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.646436 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.646849 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.646981 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.647724 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.647728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.647729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:10.647934 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.647975 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:10.647990 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.648069 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.648521 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.648605 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260809 19:34:10.654476 31663 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:10.655069 31663 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:10.664059 31663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: Generated new TSK 0
I20260809 19:34:10.664218 31663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:10.672454 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.193:0
--local_ip_for_outbound_sockets=127.31.57.193
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--builtin_ntp_servers=127.31.57.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:10.750542 31680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.750741 31680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.750780 31680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.751886 31680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.751938 31680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.751974 31680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.193
I20260809 19:34:10.753279 31680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.57.193
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.193
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.753543 31680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.753810 31680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.759258 31687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.759348 31690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.759269 31688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.759600 31680 server_base.cc:1034] running on GCE node
I20260809 19:34:10.759814 31680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:10.760085 31680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:10.761241 31680 hybrid_clock.cc:648] HybridClock initialized: now 1786304050761222 us; error 37 us; skew 500 ppm
I20260809 19:34:10.762545 31680 webserver.cc:460] Webserver started at http://127.31.57.193:39519/ using document root <none> and password file <none>
I20260809 19:34:10.762831 31680 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:10.762880 31680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:10.763023 31680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:10.764158 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data/instance:
uuid: "3036cbc276824bd89e6b4f53559d0dca"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "d3f15405126243abb02fac157f5c4ba4"
server_key_iv: "63ce93d5a1f29223b832063d0a531a1d"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.764521 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance:
uuid: "3036cbc276824bd89e6b4f53559d0dca"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "d3f15405126243abb02fac157f5c4ba4"
server_key_iv: "63ce93d5a1f29223b832063d0a531a1d"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.765866 31680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:10.766558 31696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.766767 31680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:10.766880 31680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal
uuid: "3036cbc276824bd89e6b4f53559d0dca"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "d3f15405126243abb02fac157f5c4ba4"
server_key_iv: "63ce93d5a1f29223b832063d0a531a1d"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.766957 31680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:10.785768 31680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:10.786032 31680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:10.786166 31680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:10.786368 31680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:10.786688 31680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:10.786738 31680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.786782 31680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:10.786806 31680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.792374 31680 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.193:45323
I20260809 19:34:10.792410 31881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.193:45323 every 8 connection(s)
I20260809 19:34:10.792737 31680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/data/info.pb
I20260809 19:34:10.797736 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.793473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.193:34235 (local address 127.31.57.254:37865)
0809 19:34:10.793536 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.793538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.793793 (+ 255us) server_negotiation.cc:407] Connection header received
0809 19:34:10.794295 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.794296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.794311 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.794361 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.794988 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.795552 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.796471 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.796678 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.796752 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.797462 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.797463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.797463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:10.797468 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.797483 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:10.797488 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.797534 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.797630 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.797660 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:34:10.797981 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.793466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37865 (local address 127.31.57.193:34235)
0809 19:34:10.793663 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.793692 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.794172 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.794424 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.794427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.794435 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:10.794899 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.794903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.795639 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.795640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.796379 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.796381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.796761 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.796762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.796894 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.797311 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:10.797311 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:10.797337 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:10.797357 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:10.797545 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:10.797550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:10.797552 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.797631 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.797729 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260809 19:34:10.798115 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 31680
I20260809 19:34:10.798228 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-0/wal/instance
I20260809 19:34:10.798413 31975 external_mini_cluster.cc:1428] Setting key f9db7e2f384869819a05863f5576618e
I20260809 19:34:10.798484 31882 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37865
I20260809 19:34:10.798569 31882 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:10.798743 31882 heartbeater.cc:511] Master 127.31.57.254:37865 requested a full tablet report, sending...
I20260809 19:34:10.799278 31579 ts_manager.cc:194] Registered new tserver with Master: 3036cbc276824bd89e6b4f53559d0dca (127.31.57.193:45323)
I20260809 19:34:10.799431 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.194:0
--local_ip_for_outbound_sockets=127.31.57.194
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--builtin_ntp_servers=127.31.57.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:34:10.800185 31579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.193:34235
W20260809 19:34:10.876956 31886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.877113 31886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.877131 31886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.878082 31886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.878113 31886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.878134 31886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.194
I20260809 19:34:10.879189 31886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.57.194
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.194
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.879338 31886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.879587 31886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.884123 31893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.884116 31892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.884119 31899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.884400 31886 server_base.cc:1034] running on GCE node
I20260809 19:34:10.884613 31886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:10.884847 31886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:10.885983 31886 hybrid_clock.cc:648] HybridClock initialized: now 1786304050885974 us; error 28 us; skew 500 ppm
I20260809 19:34:10.887266 31886 webserver.cc:460] Webserver started at http://127.31.57.194:45867/ using document root <none> and password file <none>
I20260809 19:34:10.887565 31886 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:10.887639 31886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:10.887782 31886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:10.888765 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data/instance:
uuid: "68dc3e90718940d7aa30d2f7b4fe031f"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "55301f8b8ce04d0ddda52c405acd1235"
server_key_iv: "b3089d1c74df5591c8a693d49ad7448c"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.889115 31886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance:
uuid: "68dc3e90718940d7aa30d2f7b4fe031f"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "55301f8b8ce04d0ddda52c405acd1235"
server_key_iv: "b3089d1c74df5591c8a693d49ad7448c"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.890372 31886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:10.891045 31906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.891222 31886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:10.891319 31886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal
uuid: "68dc3e90718940d7aa30d2f7b4fe031f"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "55301f8b8ce04d0ddda52c405acd1235"
server_key_iv: "b3089d1c74df5591c8a693d49ad7448c"
server_key_version: "encryptionkey@0"
I20260809 19:34:10.891446 31886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:10.899029 31886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:10.899287 31886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:10.899466 31886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:10.899683 31886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:10.899982 31886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:10.900036 31886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.900076 31886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:10.900102 31886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:10.905925 31886 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.194:43861
I20260809 19:34:10.905977 32096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.194:43861 every 8 connection(s)
I20260809 19:34:10.906246 31886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/data/info.pb
I20260809 19:34:10.911202 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.906917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.194:37663 (local address 127.31.57.254:37865)
0809 19:34:10.906989 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:10.906991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:10.907230 (+ 239us) server_negotiation.cc:407] Connection header received
0809 19:34:10.907841 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:10.907843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:10.907857 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:10.907914 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:10.908550 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.909053 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.909936 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.910136 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.910208 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.910947 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:10.910948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:10.910949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:10.910953 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:10.910968 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:10.910972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:10.911017 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:10.911103 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:34:10.911131 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:34:10.911450 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:10.906913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37865 (local address 127.31.57.194:37663)
0809 19:34:10.907096 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:10.907124 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:10.907683 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:10.907983 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:10.907986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:10.907993 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:10.908457 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.908461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.909153 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.909155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.909849 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:10.909851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:10.910216 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:10.910218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:10.910342 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:10.910817 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:10.910817 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:10.910841 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:10.910859 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:10.911016 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:10.911021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:10.911023 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:10.911103 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:34:10.911190 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260809 19:34:10.911903 32099 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37865
I20260809 19:34:10.911973 32099 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:10.912091 32099 heartbeater.cc:511] Master 127.31.57.254:37865 requested a full tablet report, sending...
I20260809 19:34:10.912438 31579 ts_manager.cc:194] Registered new tserver with Master: 68dc3e90718940d7aa30d2f7b4fe031f (127.31.57.194:43861)
I20260809 19:34:10.912755 31579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.194:37663
I20260809 19:34:10.914682 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 31886
I20260809 19:34:10.914762 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal/instance
I20260809 19:34:10.914899 31975 external_mini_cluster.cc:1428] Setting key 7f1a35a1a6ca6727f78f066a70e7381f
I20260809 19:34:10.915706 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.57.195:0
--local_ip_for_outbound_sockets=127.31.57.195
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--builtin_ntp_servers=127.31.57.212:36771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:34:10.991937 32102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:10.992110 32102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:10.992131 32102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:10.993300 32102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:10.993340 32102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:10.993366 32102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.57.195
I20260809 19:34:10.994721 32102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.57.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.57.195
--webserver_port=0
--tserver_master_addrs=127.31.57.254:37865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.57.195
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:10.994941 32102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:10.995213 32102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:10.999568 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:10.999612 32116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:10.999675 32102 server_base.cc:1034] running on GCE node
W20260809 19:34:10.999583 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:11.000046 32102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:11.000236 32102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:11.001410 32102 hybrid_clock.cc:648] HybridClock initialized: now 1786304051001395 us; error 37 us; skew 500 ppm
I20260809 19:34:11.002513 32102 webserver.cc:460] Webserver started at http://127.31.57.195:38751/ using document root <none> and password file <none>
I20260809 19:34:11.002738 32102 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:11.002812 32102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:11.002959 32102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:11.004017 32102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data/instance:
uuid: "b4b42b4cfdc14512adcda8068d8c64ce"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "da130f9528dfe37cd8033cf276ef0b1a"
server_key_iv: "c02c602142058d1b17d1363c3e34e06c"
server_key_version: "encryptionkey@0"
I20260809 19:34:11.004396 32102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance:
uuid: "b4b42b4cfdc14512adcda8068d8c64ce"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "da130f9528dfe37cd8033cf276ef0b1a"
server_key_iv: "c02c602142058d1b17d1363c3e34e06c"
server_key_version: "encryptionkey@0"
I20260809 19:34:11.005647 32102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:11.006325 32128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:11.006503 32102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:11.006595 32102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal
uuid: "b4b42b4cfdc14512adcda8068d8c64ce"
format_stamp: "Formatted at 2026-08-09 19:34:10 on dist-test-slave-x4qh"
server_key: "da130f9528dfe37cd8033cf276ef0b1a"
server_key_iv: "c02c602142058d1b17d1363c3e34e06c"
server_key_version: "encryptionkey@0"
I20260809 19:34:11.006671 32102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:11.028606 32102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:11.028851 32102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:11.028990 32102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:11.029184 32102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:34:11.029486 32102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:34:11.029541 32102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:11.029569 32102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:34:11.029594 32102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:11.035075 32102 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.195:40509
I20260809 19:34:11.035122 32310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.195:40509 every 8 connection(s)
I20260809 19:34:11.035408 32102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/data/info.pb
I20260809 19:34:11.039716 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.036033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:41247 (local address 127.31.57.254:37865)
0809 19:34:11.036093 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.036095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.036255 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:34:11.036663 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.036664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.036674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.036724 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:11.037291 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.037726 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.038553 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.038713 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.038758 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.039536 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:11.039537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:11.039537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:11.039541 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:11.039550 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:11.039553 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:11.039588 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.039644 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.039664 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:34:11.039932 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.036022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:37865 (local address 127.31.57.195:41247)
0809 19:34:11.036176 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.036200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.036585 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.036799 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.036802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.036809 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:11.037214 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.037218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.037802 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.037803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.038485 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.038487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.038778 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.038779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.038880 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.039354 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:34:11.039355 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:11.039432 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:34:11.039456 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:11.039589 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:11.039591 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:11.039593 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.039634 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.039701 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260809 19:34:11.040429 32312 heartbeater.cc:347] Connected to a master server at 127.31.57.254:37865
I20260809 19:34:11.040518 32312 heartbeater.cc:464] Registering TS with master...
I20260809 19:34:11.040692 32312 heartbeater.cc:511] Master 127.31.57.254:37865 requested a full tablet report, sending...
I20260809 19:34:11.041056 31579 ts_manager.cc:194] Registered new tserver with Master: b4b42b4cfdc14512adcda8068d8c64ce (127.31.57.195:40509)
I20260809 19:34:11.041076 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32102
I20260809 19:34:11.041168 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal/instance
I20260809 19:34:11.041330 31975 external_mini_cluster.cc:1428] Setting key f03925bf02f5c956f22916d85cc52130
I20260809 19:34:11.041440 31579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.57.195:41247
I20260809 19:34:11.042235 31975 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:34:11.046137 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.043634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.31.57.254:37865)
0809 19:34:11.043689 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.043691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.043787 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:34:11.043857 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.043858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.043870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.043914 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:11.044082 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.044499 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.045240 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.045400 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.045445 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.045935 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:11.045936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:11.045936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:11.045940 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:11.045950 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:11.045953 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:11.046001 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.046030 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.046046 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:34:11.047585 31579 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:34:11.048929 31579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:34:11.054739 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.051030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.31.57.195:40509)
0809 19:34:11.051183 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.051188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.051231 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:34:11.051266 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.051268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.051293 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.051388 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:11.051557 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052587 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.053629 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054265 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054310 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054383 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.054628 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.054666 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260809 19:34:11.054764 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.051123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.31.57.193:45323)
0809 19:34:11.051500 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.051504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.051528 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:34:11.051567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.051569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.051585 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.051654 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:11.051857 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052871 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054005 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054459 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054542 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054624 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.054652 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.054688 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":113,"threads_started":1}
I20260809 19:34:11.054750 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.050894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45323 (local address 127.0.0.1:52656)
0809 19:34:11.051433 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.051442 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.051483 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.051727 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.051728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.051730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:11.051780 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.051782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052951 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.052952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.053929 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.053930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054544 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054544 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.054598 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054602 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.054643 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.054691 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":89,"threads_started":1}
I20260809 19:34:11.055023 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.051145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.31.57.194:43861)
0809 19:34:11.051334 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.051338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.051505 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:34:11.051548 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.051550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.051581 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.051651 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:11.051868 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052522 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.053833 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054510 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054571 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054651 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.054901 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.054941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260809 19:34:11.054975 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.050853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.195:40509 (local address 127.0.0.1:34418)
0809 19:34:11.051035 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.051072 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.051135 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.051397 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.051400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.051402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:11.051461 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.051463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052683 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.052684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.053531 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.053534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054444 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.054516 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054529 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.054873 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.054909 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":52,"threads_started":1}
I20260809 19:34:11.055191 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.050853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43861 (local address 127.0.0.1:39256)
0809 19:34:11.051361 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.051387 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.051446 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.051721 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.051722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.051723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:11.051777 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.051779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.052610 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.052611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.053763 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.053766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.054682 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.054683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.054740 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.054743 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.054791 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.055129 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260809 19:34:11.055830 32203 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 (DEFAULT_TABLE table=test-table [id=44378af97c554875959dcc47c5a03777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:11.055841 31773 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 (DEFAULT_TABLE table=test-table [id=44378af97c554875959dcc47c5a03777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:11.055913 31992 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 (DEFAULT_TABLE table=test-table [id=44378af97c554875959dcc47c5a03777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:34:11.056151 32203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae5b16d503f4d0bad4b5b345a87ba78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:11.056175 31992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae5b16d503f4d0bad4b5b345a87ba78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:11.056178 31773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae5b16d503f4d0bad4b5b345a87ba78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:11.058926 32344 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: Bootstrap starting.
I20260809 19:34:11.059084 32345 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Bootstrap starting.
I20260809 19:34:11.059276 32346 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Bootstrap starting.
I20260809 19:34:11.059711 32345 tablet_bootstrap.cc:654] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:11.059710 32344 tablet_bootstrap.cc:654] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:11.059840 32346 tablet_bootstrap.cc:654] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:11.060017 32344 log.cc:824] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:11.060017 32345 log.cc:824] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:11.060107 32346 log.cc:824] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:11.060702 32345 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: No bootstrap required, opened a new log
I20260809 19:34:11.060750 32345 ts_tablet_manager.cc:1397] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:34:11.060752 32346 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: No bootstrap required, opened a new log
I20260809 19:34:11.060813 32346 ts_tablet_manager.cc:1397] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:34:11.060899 32344 tablet_bootstrap.cc:492] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: No bootstrap required, opened a new log
I20260809 19:34:11.060945 32344 ts_tablet_manager.cc:1397] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:34:11.062515 32345 raft_consensus.cc:348] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.062659 32345 raft_consensus.cc:374] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:11.062718 32345 raft_consensus.cc:729] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b42b4cfdc14512adcda8068d8c64ce, State: Initialized, Role: FOLLOWER
I20260809 19:34:11.062839 32345 consensus_queue.cc:260] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.063104 32345 ts_tablet_manager.cc:1428] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:34:11.063125 32312 heartbeater.cc:503] Master 127.31.57.254:37865 was elected leader, sending a full tablet report...
I20260809 19:34:11.063241 32346 raft_consensus.cc:348] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.063274 32344 raft_consensus.cc:348] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.063426 32346 raft_consensus.cc:374] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:11.063443 32344 raft_consensus.cc:374] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:11.063482 32346 raft_consensus.cc:729] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68dc3e90718940d7aa30d2f7b4fe031f, State: Initialized, Role: FOLLOWER
I20260809 19:34:11.063493 32344 raft_consensus.cc:729] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3036cbc276824bd89e6b4f53559d0dca, State: Initialized, Role: FOLLOWER
I20260809 19:34:11.063580 32344 consensus_queue.cc:260] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.063617 32346 consensus_queue.cc:260] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.063820 32346 ts_tablet_manager.cc:1428] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:34:11.063818 32344 ts_tablet_manager.cc:1428] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:34:11.063925 32099 heartbeater.cc:503] Master 127.31.57.254:37865 was elected leader, sending a full tablet report...
I20260809 19:34:11.063936 31882 heartbeater.cc:503] Master 127.31.57.254:37865 was elected leader, sending a full tablet report...
I20260809 19:34:11.127252 32352 raft_consensus.cc:482] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:34:11.127344 32352 raft_consensus.cc:504] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.127912 32352 leader_election.cc:288] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68dc3e90718940d7aa30d2f7b4fe031f (127.31.57.194:43861), 3036cbc276824bd89e6b4f53559d0dca (127.31.57.193:45323)
I20260809 19:34:11.130410 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.128050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.194:43861 (local address 127.31.57.195:38279)
0809 19:34:11.128098 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.128110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.128161 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.128285 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.128286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.128287 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:11.128352 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.128354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.128951 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.128952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.129610 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.129611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.130170 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.130171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.130244 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.130248 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.130305 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.130345 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:34:11.130439 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.128148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:38279 (local address 127.31.57.194:43861)
0809 19:34:11.128187 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.128189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.128198 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:11.128224 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.128224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.128232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.128274 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:11.128419 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.128856 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.129715 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.130085 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.130166 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.130175 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.130331 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.130372 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:34:11.130661 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.128127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.57.195:42271 (local address 127.31.57.193:45323)
0809 19:34:11.128168 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.128170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.128334 (+ 164us) server_negotiation.cc:407] Connection header received
0809 19:34:11.128404 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.128404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.128426 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.128485 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:34:11.128631 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.129035 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.129878 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.130275 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.130341 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.130347 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.130551 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.130594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:34:11.130795 32022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae5b16d503f4d0bad4b5b345a87ba78" candidate_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" is_pre_election: true
I20260809 19:34:11.130884 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.128050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45323 (local address 127.31.57.195:42271)
0809 19:34:11.128261 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.128269 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:11.128342 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:11.128488 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:11.128489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:11.128490 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:34:11.128558 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.128560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.129119 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.129119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.129771 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:11.129773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.130358 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.130359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:11.130428 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.130432 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:34:11.130742 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 19:34:11.130789 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":42,"thread_start_us":103,"threads_started":1}
I20260809 19:34:11.130939 32022 raft_consensus.cc:2454] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4b42b4cfdc14512adcda8068d8c64ce in term 0.
I20260809 19:34:11.131134 32131 leader_election.cc:302] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68dc3e90718940d7aa30d2f7b4fe031f, b4b42b4cfdc14512adcda8068d8c64ce; no voters:
I20260809 19:34:11.131203 31808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae5b16d503f4d0bad4b5b345a87ba78" candidate_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3036cbc276824bd89e6b4f53559d0dca" is_pre_election: true
I20260809 19:34:11.131276 32352 raft_consensus.cc:2781] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:34:11.131387 32352 raft_consensus.cc:482] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:34:11.131353 31808 raft_consensus.cc:2454] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4b42b4cfdc14512adcda8068d8c64ce in term 0.
I20260809 19:34:11.131418 32352 raft_consensus.cc:3037] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:11.132133 32352 raft_consensus.cc:504] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.132280 32352 leader_election.cc:288] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [CANDIDATE]: Term 1 election: Requested vote from peers 68dc3e90718940d7aa30d2f7b4fe031f (127.31.57.194:43861), 3036cbc276824bd89e6b4f53559d0dca (127.31.57.193:45323)
I20260809 19:34:11.132433 31808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae5b16d503f4d0bad4b5b345a87ba78" candidate_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3036cbc276824bd89e6b4f53559d0dca"
I20260809 19:34:11.132465 32022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae5b16d503f4d0bad4b5b345a87ba78" candidate_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68dc3e90718940d7aa30d2f7b4fe031f"
I20260809 19:34:11.132519 31808 raft_consensus.cc:3037] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:11.132529 32022 raft_consensus.cc:3037] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:11.133213 31808 raft_consensus.cc:2454] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b42b4cfdc14512adcda8068d8c64ce in term 1.
I20260809 19:34:11.133256 32022 raft_consensus.cc:2454] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b42b4cfdc14512adcda8068d8c64ce in term 1.
I20260809 19:34:11.133450 32135 leader_election.cc:302] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3036cbc276824bd89e6b4f53559d0dca, b4b42b4cfdc14512adcda8068d8c64ce; no voters:
I20260809 19:34:11.133546 32352 raft_consensus.cc:2781] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:11.133718 32352 raft_consensus.cc:686] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 1 LEADER]: Becoming Leader. State: Replica: b4b42b4cfdc14512adcda8068d8c64ce, State: Running, Role: LEADER
I20260809 19:34:11.133814 32352 consensus_queue.cc:237] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } }
I20260809 19:34:11.134557 31578 catalog_manager.cc:5495] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce reported cstate change: term changed from 0 to 1, leader changed from <none> to b4b42b4cfdc14512adcda8068d8c64ce (127.31.57.195). New cstate: current_term: 1 leader_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b42b4cfdc14512adcda8068d8c64ce" member_type: VOTER last_known_addr { host: "127.31.57.195" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260809 19:34:11.146060 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.143846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.31.57.195:40509)
0809 19:34:11.143880 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:11.143882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:11.143889 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:11.143915 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:11.143916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:11.143930 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:11.144002 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:11.144144 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.144580 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.145318 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:11.145514 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:11.145594 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:11.145760 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:11.145861 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:11.145948 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:11.145966 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 19:34:11.145982 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:11.147572 31808 raft_consensus.cc:1261] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 1 FOLLOWER]: Refusing update from remote peer b4b42b4cfdc14512adcda8068d8c64ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:11.147572 32022 raft_consensus.cc:1261] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 1 FOLLOWER]: Refusing update from remote peer b4b42b4cfdc14512adcda8068d8c64ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:11.147876 32352 consensus_queue.cc:1035] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "3036cbc276824bd89e6b4f53559d0dca" member_type: VOTER last_known_addr { host: "127.31.57.193" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:11.148003 32361 consensus_queue.cc:1035] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "68dc3e90718940d7aa30d2f7b4fe031f" member_type: VOTER last_known_addr { host: "127.31.57.194" port: 43861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:11.150748 32372 mvcc.cc:205] Tried to move back new op lower bound from 7316701393498038272 to 7316701393444737024. Current Snapshot: MvccSnapshot[applied={T|T < 7316701393498038272 or (T in {7316701393498038272})}]
I20260809 19:34:11.151072 32369 mvcc.cc:205] Tried to move back new op lower bound from 7316701393498038272 to 7316701393444737024. Current Snapshot: MvccSnapshot[applied={T|T < 7316701393498038272}]
I20260809 19:34:11.152302 31579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:11.152395 31579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:34:11.153079 31579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c705c04a1e714c3e807f34cac3c49e5c: Sending DeleteTablet for 3 replicas of tablet 8ae5b16d503f4d0bad4b5b345a87ba78
I20260809 19:34:11.153368 32203 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:11 UTC) from {username='slave'} at 127.0.0.1:34418
I20260809 19:34:11.153465 31992 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:11 UTC) from {username='slave'} at 127.0.0.1:39256
I20260809 19:34:11.153507 31773 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae5b16d503f4d0bad4b5b345a87ba78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:34:11 UTC) from {username='slave'} at 127.0.0.1:52656
I20260809 19:34:11.153712 32384 tablet_replica.cc:330] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: stopping tablet replica
I20260809 19:34:11.153868 32385 tablet_replica.cc:330] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca: stopping tablet replica
I20260809 19:34:11.153889 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 31680
I20260809 19:34:11.153963 32385 raft_consensus.cc:2229] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:11.154053 32385 raft_consensus.cc:2258] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 3036cbc276824bd89e6b4f53559d0dca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:11.154192 32383 tablet_replica.cc:330] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: stopping tablet replica
I20260809 19:34:11.154268 32383 raft_consensus.cc:2229] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:34:11.156718 32383 raft_consensus.cc:2258] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:11.159531 32383 ts_tablet_manager.cc:1905] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:11.153932 32384 raft_consensus.cc:2229] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:34:11.161119 32384 raft_consensus.cc:2258] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:34:11.161968 32383 ts_tablet_manager.cc:1918] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:11.162048 32383 log.cc:1196] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-2/wal/wals/8ae5b16d503f4d0bad4b5b345a87ba78
I20260809 19:34:11.162375 32383 ts_tablet_manager.cc:1939] T 8ae5b16d503f4d0bad4b5b345a87ba78 P b4b42b4cfdc14512adcda8068d8c64ce: Deleting consensus metadata
I20260809 19:34:11.163148 31547 catalog_manager.cc:4841] TS b4b42b4cfdc14512adcda8068d8c64ce (127.31.57.195:40509): tablet 8ae5b16d503f4d0bad4b5b345a87ba78 (table test-table [id=44378af97c554875959dcc47c5a03777]) successfully deleted
I20260809 19:34:11.163915 32384 ts_tablet_manager.cc:1905] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:34:11.165490 32384 ts_tablet_manager.cc:1918] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:34:11.165555 32384 log.cc:1196] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Deleting WAL directory at /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303811347120-31975-0/minicluster-data/ts-1/wal/wals/8ae5b16d503f4d0bad4b5b345a87ba78
I20260809 19:34:11.165798 32384 ts_tablet_manager.cc:1939] T 8ae5b16d503f4d0bad4b5b345a87ba78 P 68dc3e90718940d7aa30d2f7b4fe031f: Deleting consensus metadata
I20260809 19:34:11.171834 31548 catalog_manager.cc:4841] TS 68dc3e90718940d7aa30d2f7b4fe031f (127.31.57.194:43861): tablet 8ae5b16d503f4d0bad4b5b345a87ba78 (table test-table [id=44378af97c554875959dcc47c5a03777]) successfully deleted
W20260809 19:34:11.180120 31552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.57.193:45323 (error 108)
I20260809 19:34:11.180321 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 31886
I20260809 19:34:11.187426 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:11.185184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.193:45323 (local address 127.0.0.1:52660)
0809 19:34:11.187291 (+ 2107us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:11.187365 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2069}
W20260809 19:34:11.187628 31552 catalog_manager.cc:4584] TS 3036cbc276824bd89e6b4f53559d0dca (127.31.57.193:45323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ae5b16d503f4d0bad4b5b345a87ba78: Network error: Client connection negotiation failed: client connection to 127.31.57.193:45323: connect: Connection refused (error 111)
I20260809 19:34:11.192405 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32102
I20260809 19:34:11.203511 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 31523
2026-08-09T19:34:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (704 ms)
[----------] 6 tests from EncryptionPolicyTest (8735 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:11 dist-test-slave-x4qh krb5kdc[32389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:11 dist-test-slave-x4qh krb5kdc[32389](info): set up 2 sockets
Aug 09 19:34:11 dist-test-slave-x4qh krb5kdc[32389](info): commencing operation
krb5kdc: starting...
W20260809 19:34:12.394862 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:12.507706 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:12.584852 32410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:12.585078 32410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:12.585114 32410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:12.586179 32410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:12.586246 32410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:12.586267 32410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:12.586285 32410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:12.586309 32410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:12.587765 32410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:12.588042 32410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:12.588261 32410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:12.593101 32421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:12.593086 32427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:12.593065 32422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:12.593271 32410 server_base.cc:1034] running on GCE node
I20260809 19:34:12.593467 32410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:12.593761 32410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:12.594905 32410 hybrid_clock.cc:648] HybridClock initialized: now 1786304052594892 us; error 30 us; skew 500 ppm
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:12.597538 32410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:12.597945 32410 webserver.cc:460] Webserver started at http://127.0.0.1:32875/ using document root <none> and password file <none>
I20260809 19:34:12.598173 32410 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:12.598241 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:12.598393 32410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:12.599491 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "61a61961a61a4eac98ae6335290a56e2"
format_stamp: "Formatted at 2026-08-09 19:34:12 on dist-test-slave-x4qh"
server_key: "e5a441440a213fc59931af6f4a6801f2"
server_key_iv: "62a849a7ed58a2e346fea635387c7574"
server_key_version: "encryptionkey@0"
I20260809 19:34:12.599869 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "61a61961a61a4eac98ae6335290a56e2"
format_stamp: "Formatted at 2026-08-09 19:34:12 on dist-test-slave-x4qh"
server_key: "e5a441440a213fc59931af6f4a6801f2"
server_key_iv: "62a849a7ed58a2e346fea635387c7574"
server_key_version: "encryptionkey@0"
I20260809 19:34:12.601189 32410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:12.601902 32443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:12.602070 32410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:12.602169 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "61a61961a61a4eac98ae6335290a56e2"
format_stamp: "Formatted at 2026-08-09 19:34:12 on dist-test-slave-x4qh"
server_key: "e5a441440a213fc59931af6f4a6801f2"
server_key_iv: "62a849a7ed58a2e346fea635387c7574"
server_key_version: "encryptionkey@0"
I20260809 19:34:12.602248 32410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:12.618986 32410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:12.621228 32410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:12.621393 32410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:12.625342 32410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42231
I20260809 19:34:12.625393 32526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42231 every 8 connection(s)
I20260809 19:34:12.625744 32410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:12.626646 32527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:12.628947 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: Bootstrap starting.
I20260809 19:34:12.629559 32527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:12.629870 32527 log.cc:824] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:12.630564 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: No bootstrap required, opened a new log
I20260809 19:34:12.632124 32527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260809 19:34:12.632261 32527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:12.632247 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32410
I20260809 19:34:12.632320 32527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a61961a61a4eac98ae6335290a56e2, State: Initialized, Role: FOLLOWER
I20260809 19:34:12.632395 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:12.632464 32527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260809 19:34:12.632534 32527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:12.632566 32527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:12.632606 31975 external_mini_cluster.cc:1428] Setting key cf8e6b6e200b15efb31b854560422bd8
I20260809 19:34:12.632611 32527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:12.633298 32527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260809 19:34:12.633436 32527 leader_election.cc:302] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61a61961a61a4eac98ae6335290a56e2; no voters:
I20260809 19:34:12.633615 32527 leader_election.cc:288] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:12.633781 32536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:12.633877 32527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:12.633934 32536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [term 1 LEADER]: Becoming Leader. State: Replica: 61a61961a61a4eac98ae6335290a56e2, State: Running, Role: LEADER
I20260809 19:34:12.634023 32536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260809 19:34:12.634359 32536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61a61961a61a4eac98ae6335290a56e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } } }
I20260809 19:34:12.634438 32536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:12.634575 32536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61a61961a61a4eac98ae6335290a56e2. Latest consensus state: current_term: 1 leader_uuid: "61a61961a61a4eac98ae6335290a56e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61a61961a61a4eac98ae6335290a56e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } } }
I20260809 19:34:12.634640 32536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2 [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:12.635876 32561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:12.635936 32561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:12.635991 32547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:12.636142 32547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:12.637733 32547 catalog_manager.cc:1347] Generated new cluster ID: 7f4d340976554e9c81f151bd8c46133b
I20260809 19:34:12.637786 32547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:34:12.643599 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:12.633117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52123 (local address 127.0.0.1:42231)
0809 19:34:12.633299 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:12.633308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:12.633332 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:34:12.633445 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:12.633452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:12.633509 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:12.633647 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:12.634511 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:12.636280 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:12.637244 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:12.637843 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:12.637951 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:12.640338 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:12.640342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:12.640342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:12.640353 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:12.641980 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:12.642381 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:12.642384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:12.642384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:12.642524 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:12.642766 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:12.642768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:12.642768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:12.642994 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:12.643181 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:12.643266 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 19:34:12.643423 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260809 19:34:12.644701 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:12.648074 32547 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:12.648749 32547 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:12.652594 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:12.646069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60275 (local address 127.0.0.1:42231)
0809 19:34:12.646108 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:12.646110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:12.646241 (+ 131us) server_negotiation.cc:407] Connection header received
0809 19:34:12.646311 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:12.646312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:12.646321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:12.646376 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:12.646869 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:12.647492 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:12.648483 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:12.648735 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:12.648845 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:12.650034 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:12.650035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:12.650035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:12.650042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:12.651419 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:12.651760 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:12.651761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:12.651762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:12.651831 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:12.652051 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:12.652052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:12.652052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:12.652157 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:12.652273 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:12.652423 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 19:34:12.652486 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:12.655364 32547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61a61961a61a4eac98ae6335290a56e2: Generated new TSK 0
I20260809 19:34:12.655560 32547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:12.658931 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32410
W20260809 19:34:12.668920 32387 connection.cc:535] client connection to 127.0.0.1:42231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42231): Connection reset by peer (error 104)
2026-08-09T19:34:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32589](info): set up 2 sockets
Aug 09 19:34:12 dist-test-slave-x4qh krb5kdc[32589](info): commencing operation
krb5kdc: starting...
W20260809 19:34:13.842710 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:13 dist-test-slave-x4qh krb5kdc[32589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:13.954237 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:14.032136 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:14.032305 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:14.032325 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:14.033458 32610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:14.033499 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:14.033514 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:14.033553 32610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:14.033583 32610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:14.034922 32610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:14.035158 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:14.035363 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:14.039842 32623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:14.039834 32617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:14.039921 32618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:14.039978 32610 server_base.cc:1034] running on GCE node
I20260809 19:34:14.040297 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:14.040596 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:14.041734 32610 hybrid_clock.cc:648] HybridClock initialized: now 1786304054041719 us; error 32 us; skew 500 ppm
Aug 09 19:34:14 dist-test-slave-x4qh krb5kdc[32589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304054, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:14.044082 32610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:14.044471 32610 webserver.cc:460] Webserver started at http://127.0.0.1:37881/ using document root <none> and password file <none>
I20260809 19:34:14.044656 32610 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:14.044693 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:14.044778 32610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:14.045672 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "701db3b7d570423ebfb345cd717df440"
format_stamp: "Formatted at 2026-08-09 19:34:14 on dist-test-slave-x4qh"
server_key: "394d8e1c1c3c19bb4162e10dbd50c76e"
server_key_iv: "2a32646c5f3090967348e9b4e9248883"
server_key_version: "encryptionkey@0"
I20260809 19:34:14.045962 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "701db3b7d570423ebfb345cd717df440"
format_stamp: "Formatted at 2026-08-09 19:34:14 on dist-test-slave-x4qh"
server_key: "394d8e1c1c3c19bb4162e10dbd50c76e"
server_key_iv: "2a32646c5f3090967348e9b4e9248883"
server_key_version: "encryptionkey@0"
I20260809 19:34:14.047083 32610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:14.047780 32631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:14.047919 32610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:14.048018 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "701db3b7d570423ebfb345cd717df440"
format_stamp: "Formatted at 2026-08-09 19:34:14 on dist-test-slave-x4qh"
server_key: "394d8e1c1c3c19bb4162e10dbd50c76e"
server_key_iv: "2a32646c5f3090967348e9b4e9248883"
server_key_version: "encryptionkey@0"
I20260809 19:34:14.048085 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:14.059109 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:14.060700 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:14.060835 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:14.064559 32610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41075
I20260809 19:34:14.064625 32725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41075 every 8 connection(s)
I20260809 19:34:14.064941 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:14.065755 32726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:14.067855 32726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440: Bootstrap starting.
I20260809 19:34:14.068336 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 32610
I20260809 19:34:14.068483 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:14.068537 32726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:14.068702 31975 external_mini_cluster.cc:1428] Setting key 1367a436361633916b48cb27977aed44
I20260809 19:34:14.068894 32726 log.cc:824] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:14.069620 32726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440: No bootstrap required, opened a new log
I20260809 19:34:14.071720 32726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } }
I20260809 19:34:14.071875 32726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:14.071906 32726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701db3b7d570423ebfb345cd717df440, State: Initialized, Role: FOLLOWER
I20260809 19:34:14.072016 32726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } }
I20260809 19:34:14.072082 32726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:14.072110 32726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:14.072155 32726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:14.072708 32726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } }
I20260809 19:34:14.072817 32726 leader_election.cc:302] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 701db3b7d570423ebfb345cd717df440; no voters:
I20260809 19:34:14.072993 32726 leader_election.cc:288] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:14.073026 32737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:14.073180 32737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [term 1 LEADER]: Becoming Leader. State: Replica: 701db3b7d570423ebfb345cd717df440, State: Running, Role: LEADER
I20260809 19:34:14.073298 32726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:14.073307 32737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } }
I20260809 19:34:14.073678 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 701db3b7d570423ebfb345cd717df440. Latest consensus state: current_term: 1 leader_uuid: "701db3b7d570423ebfb345cd717df440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } } }
I20260809 19:34:14.073691 32739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "701db3b7d570423ebfb345cd717df440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701db3b7d570423ebfb345cd717df440" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41075 } } }
I20260809 19:34:14.073868 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:14.073877 32739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:14.074301 32752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:14.074453 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:14.069179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51753 (local address 127.0.0.1:41075)
0809 19:34:14.069366 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:14.069374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:14.069407 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:34:14.069883 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:14.069889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:14.069932 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:14.070034 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:14.070283 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:14.071081 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:14.072214 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:14.072624 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:14.072702 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:14.073640 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:14.073643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:14.073644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:14.073842 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:14.073877 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:14.073891 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:14.073968 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:14.074176 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:34:14.074202 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260809 19:34:14.075017 32752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:14.075428 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:14.077047 32752 catalog_manager.cc:1347] Generated new cluster ID: d2c249e17dfd43e8a190d71d494558d7
I20260809 19:34:14.077098 32752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:14.079888 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:14.076699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39811 (local address 127.0.0.1:41075)
0809 19:34:14.076749 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:14.076751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:14.076837 (+ 86us) server_negotiation.cc:407] Connection header received
0809 19:34:14.076895 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:14.076896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:14.076903 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:14.076942 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:14.077086 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:14.077709 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:14.078616 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:14.078915 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:14.078946 (+ 31us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:14.079656 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:14.079657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:14.079657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:14.079661 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:14.079671 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:14.079675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:14.079701 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:14.079817 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:34:14.079826 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:34:14.086045 32752 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:14.086684 32752 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:14.089794 32752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 701db3b7d570423ebfb345cd717df440: Generated new TSK 0
I20260809 19:34:14.089949 32752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:14.093528 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 32610
W20260809 19:34:14.112792 32586 connection.cc:535] client connection to 127.0.0.1:41075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41075): Connection reset by peer (error 104)
2026-08-09T19:34:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:14 dist-test-slave-x4qh krb5kdc[316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:14 dist-test-slave-x4qh krb5kdc[316](info): set up 2 sockets
Aug 09 19:34:14 dist-test-slave-x4qh krb5kdc[316](info): commencing operation
krb5kdc: starting...
W20260809 19:34:15.279748 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:15 dist-test-slave-x4qh krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:15.392412 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:15.469995 339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:15.470167 339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:15.470207 339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:15.471220 339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:15.471253 339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:15.471267 339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:15.471287 339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:15.471298 339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:15.472649 339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37373
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:15.472839 339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:15.472997 339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:15.477514 349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:15.477514 347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:15.477586 339 server_base.cc:1034] running on GCE node
W20260809 19:34:15.477569 346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:15.478092 339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:15.478328 339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:15.479483 339 hybrid_clock.cc:648] HybridClock initialized: now 1786304055479482 us; error 36 us; skew 500 ppm
Aug 09 19:34:15 dist-test-slave-x4qh krb5kdc[316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304055, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:15.486253 339 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:15.486743 339 webserver.cc:460] Webserver started at http://127.0.0.1:42759/ using document root <none> and password file <none>
I20260809 19:34:15.486989 339 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:15.487057 339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:15.487197 339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:15.488220 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "624bdf68366545c3a927ed61ff30d623"
format_stamp: "Formatted at 2026-08-09 19:34:15 on dist-test-slave-x4qh"
server_key: "0141212c30bea20d8d2b43d5e763a777"
server_key_iv: "79f6cdf8549f236656d251db6f4ee35e"
server_key_version: "encryptionkey@0"
I20260809 19:34:15.488560 339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "624bdf68366545c3a927ed61ff30d623"
format_stamp: "Formatted at 2026-08-09 19:34:15 on dist-test-slave-x4qh"
server_key: "0141212c30bea20d8d2b43d5e763a777"
server_key_iv: "79f6cdf8549f236656d251db6f4ee35e"
server_key_version: "encryptionkey@0"
I20260809 19:34:15.489817 339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:15.490522 363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:15.490686 339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:15.490796 339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "624bdf68366545c3a927ed61ff30d623"
format_stamp: "Formatted at 2026-08-09 19:34:15 on dist-test-slave-x4qh"
server_key: "0141212c30bea20d8d2b43d5e763a777"
server_key_iv: "79f6cdf8549f236656d251db6f4ee35e"
server_key_version: "encryptionkey@0"
I20260809 19:34:15.490895 339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:15.494118 339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:15.495927 339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:15.496084 339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:15.500205 339 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37373
I20260809 19:34:15.500236 455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37373 every 8 connection(s)
I20260809 19:34:15.500543 339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:15.501384 457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:15.503592 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623: Bootstrap starting.
I20260809 19:34:15.504180 457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:15.504516 457 log.cc:824] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:15.505244 457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623: No bootstrap required, opened a new log
I20260809 19:34:15.506907 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 339
I20260809 19:34:15.507026 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:15.506947 457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } }
I20260809 19:34:15.507159 457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:15.507221 31975 external_mini_cluster.cc:1428] Setting key 2b6b0b061a948827a70169ffcd498d5d
I20260809 19:34:15.507213 457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624bdf68366545c3a927ed61ff30d623, State: Initialized, Role: FOLLOWER
I20260809 19:34:15.507349 457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } }
I20260809 19:34:15.507499 457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:15.507560 457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:15.507637 457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:15.508965 457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } }
I20260809 19:34:15.509094 457 leader_election.cc:302] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 624bdf68366545c3a927ed61ff30d623; no voters:
I20260809 19:34:15.509254 457 leader_election.cc:288] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:15.509420 466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:15.509516 457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:15.509541 466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [term 1 LEADER]: Becoming Leader. State: Replica: 624bdf68366545c3a927ed61ff30d623, State: Running, Role: LEADER
I20260809 19:34:15.509613 466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } }
I20260809 19:34:15.509950 466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "624bdf68366545c3a927ed61ff30d623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } } }
I20260809 19:34:15.510035 466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:15.510166 466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 624bdf68366545c3a927ed61ff30d623. Latest consensus state: current_term: 1 leader_uuid: "624bdf68366545c3a927ed61ff30d623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "624bdf68366545c3a927ed61ff30d623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37373 } } }
I20260809 19:34:15.510218 466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:15.510196 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:15.507740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51483 (local address 127.0.0.1:37373)
0809 19:34:15.507951 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:15.507960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:15.508049 (+ 89us) server_negotiation.cc:407] Connection header received
0809 19:34:15.508246 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:15.508254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:15.508323 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:15.508524 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:15.509144 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:15.509148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:15.509148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:15.509153 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:15.509205 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:15.509222 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:15.509303 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:15.509580 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 19:34:15.509768 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260809 19:34:15.510980 474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:15.512239 474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:15.512470 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:15.515864 474 catalog_manager.cc:1347] Generated new cluster ID: ce68b14eb0784f168aa649170a6434f3
I20260809 19:34:15.515920 474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:15.516894 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:15.513952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56997 (local address 127.0.0.1:37373)
0809 19:34:15.514170 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:15.514172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:15.515978 (+ 1806us) server_negotiation.cc:407] Connection header received
0809 19:34:15.516064 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:15.516065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:15.516070 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:15.516136 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:15.516683 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:15.516684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:15.516684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:15.516688 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:15.516703 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:15.516707 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:15.516769 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:15.516819 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:34:15.516831 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260809 19:34:15.536685 474 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:15.536844 474 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:15.544548 474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 624bdf68366545c3a927ed61ff30d623: Generated new TSK 0
I20260809 19:34:15.544715 474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:15.547544 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 339
2026-08-09T19:34:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:15 dist-test-slave-x4qh krb5kdc[508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:15 dist-test-slave-x4qh krb5kdc[508](info): set up 2 sockets
Aug 09 19:34:15 dist-test-slave-x4qh krb5kdc[508](info): commencing operation
krb5kdc: starting...
W20260809 19:34:16.788432 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:16 dist-test-slave-x4qh krb5kdc[508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:16.893982 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:16.969971 525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:16.970203 525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:16.970240 525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:16.971299 525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:16.971360 525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:16.971380 525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:16.971477 525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:16.971505 525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:16.972856 525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:16.973115 525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:16.973325 525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:16.978149 536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:16.978189 525 server_base.cc:1034] running on GCE node
W20260809 19:34:16.978163 533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:16.978166 530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:16.978586 525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:16.978904 525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:16.980053 525 hybrid_clock.cc:648] HybridClock initialized: now 1786304056980030 us; error 37 us; skew 500 ppm
Aug 09 19:34:16 dist-test-slave-x4qh krb5kdc[508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304056, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:16.982388 525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:16.982831 525 webserver.cc:460] Webserver started at http://127.0.0.1:36549/ using document root <none> and password file <none>
I20260809 19:34:16.983008 525 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:16.983048 525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:16.983132 525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:16.984125 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "48a0d0063bc54d78a326f2ca0f010d1b"
format_stamp: "Formatted at 2026-08-09 19:34:16 on dist-test-slave-x4qh"
server_key: "86e4fce9e286f02d7b25dfd08834242b"
server_key_iv: "4e312cfa80ffda5b920f168322246bae"
server_key_version: "encryptionkey@0"
I20260809 19:34:16.984424 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "48a0d0063bc54d78a326f2ca0f010d1b"
format_stamp: "Formatted at 2026-08-09 19:34:16 on dist-test-slave-x4qh"
server_key: "86e4fce9e286f02d7b25dfd08834242b"
server_key_iv: "4e312cfa80ffda5b920f168322246bae"
server_key_version: "encryptionkey@0"
I20260809 19:34:16.985589 525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:16.986248 548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:16.986418 525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:16.986521 525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "48a0d0063bc54d78a326f2ca0f010d1b"
format_stamp: "Formatted at 2026-08-09 19:34:16 on dist-test-slave-x4qh"
server_key: "86e4fce9e286f02d7b25dfd08834242b"
server_key_iv: "4e312cfa80ffda5b920f168322246bae"
server_key_version: "encryptionkey@0"
I20260809 19:34:16.986593 525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:16.996766 525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:16.998431 525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:16.998584 525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:17.002601 525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42349
I20260809 19:34:17.002630 647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42349 every 8 connection(s)
I20260809 19:34:17.002960 525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:17.003810 648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:17.005878 648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: Bootstrap starting.
I20260809 19:34:17.006398 648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:17.006628 648 log.cc:824] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:17.007236 648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: No bootstrap required, opened a new log
I20260809 19:34:17.008167 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 525
I20260809 19:34:17.008291 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:17.008443 31975 external_mini_cluster.cc:1428] Setting key acced6c3c8acda07510ff5faa21e0e01
I20260809 19:34:17.009269 648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } }
I20260809 19:34:17.009454 648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:17.009502 648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a0d0063bc54d78a326f2ca0f010d1b, State: Initialized, Role: FOLLOWER
I20260809 19:34:17.009658 648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } }
I20260809 19:34:17.009753 648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:17.009788 648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:17.009837 648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:17.010840 648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } }
I20260809 19:34:17.010950 648 leader_election.cc:302] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a0d0063bc54d78a326f2ca0f010d1b; no voters:
I20260809 19:34:17.011091 648 leader_election.cc:288] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:17.011134 657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:17.011276 657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [term 1 LEADER]: Becoming Leader. State: Replica: 48a0d0063bc54d78a326f2ca0f010d1b, State: Running, Role: LEADER
I20260809 19:34:17.011319 648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:17.011480 657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } }
I20260809 19:34:17.012665 660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a0d0063bc54d78a326f2ca0f010d1b. Latest consensus state: current_term: 1 leader_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } } }
I20260809 19:34:17.012840 659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a0d0063bc54d78a326f2ca0f010d1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42349 } } }
W20260809 19:34:17.013182 679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 19:34:17.013684 660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:17.013697 659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:17.013713 679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:17.013949 683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:17.014110 683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:34:17 dist-test-slave-x4qh krb5kdc[508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:34:17.015733 683 catalog_manager.cc:1347] Generated new cluster ID: 6b16fa5ba16148ddb5bd53de6ece1118
I20260809 19:34:17.015797 683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:17.019220 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:17.008905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60361 (local address 127.0.0.1:42349)
0809 19:34:17.009114 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:17.009124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:17.009156 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:34:17.009261 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:17.009268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:17.009317 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:17.009473 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:17.010242 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:17.011648 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:17.012709 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:17.013315 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:17.013396 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:17.016118 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:17.016122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:17.016122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:17.016134 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:17.017675 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:17.018065 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:17.018068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:17.018068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:17.018186 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:17.018395 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:17.018396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:17.018396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:17.018614 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:17.018796 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:17.018899 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 19:34:17.019034 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260809 19:34:17.020066 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:17.020840 683 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:17.021531 683 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:17.025511 683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a0d0063bc54d78a326f2ca0f010d1b: Generated new TSK 0
I20260809 19:34:17.025705 683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:17.028919 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:17.021637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32797 (local address 127.0.0.1:42349)
0809 19:34:17.021672 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:17.021674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:17.023573 (+ 1899us) server_negotiation.cc:407] Connection header received
0809 19:34:17.023645 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:17.023647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:17.023656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:17.023711 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:17.024209 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:17.024673 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:17.025578 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:17.025782 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:17.025852 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:17.026864 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:17.026866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:17.026866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:17.026872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:17.027962 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:17.028257 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:17.028258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:17.028258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:17.028309 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:17.028498 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:17.028499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:17.028499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:17.028587 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:17.028671 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:17.028765 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:34:17.028840 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:17.030491 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 525
2026-08-09T19:34:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:17 dist-test-slave-x4qh krb5kdc[699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:17 dist-test-slave-x4qh krb5kdc[699](info): set up 2 sockets
Aug 09 19:34:17 dist-test-slave-x4qh krb5kdc[699](info): commencing operation
krb5kdc: starting...
W20260809 19:34:18.202070 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:18 dist-test-slave-x4qh krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:18.308210 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:18.385397 720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:18.385627 720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:18.385666 720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:18.386761 720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:18.386827 720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:18.386852 720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:18.386879 720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:18.386904 720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:18.388398 720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:18.388662 720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:18.388888 720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:18.393658 728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:18.393723 729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:18.393807 720 server_base.cc:1034] running on GCE node
W20260809 19:34:18.393644 733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:18.394080 720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:18.394414 720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:18.395557 720 hybrid_clock.cc:648] HybridClock initialized: now 1786304058395532 us; error 38 us; skew 500 ppm
Aug 09 19:34:18 dist-test-slave-x4qh krb5kdc[699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304058, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:18.397838 720 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:18.398248 720 webserver.cc:460] Webserver started at http://127.0.0.1:32891/ using document root <none> and password file <none>
I20260809 19:34:18.398494 720 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:18.398562 720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:18.398712 720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:18.399773 720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "39218fa275234524850eed75ef7b98fd"
format_stamp: "Formatted at 2026-08-09 19:34:18 on dist-test-slave-x4qh"
server_key: "22004e2d9cf737b79768093a80e43469"
server_key_iv: "d8f86aa0cceb3fca58708c75e92f2417"
server_key_version: "encryptionkey@0"
I20260809 19:34:18.400120 720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "39218fa275234524850eed75ef7b98fd"
format_stamp: "Formatted at 2026-08-09 19:34:18 on dist-test-slave-x4qh"
server_key: "22004e2d9cf737b79768093a80e43469"
server_key_iv: "d8f86aa0cceb3fca58708c75e92f2417"
server_key_version: "encryptionkey@0"
I20260809 19:34:18.401295 720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:18.401916 750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:18.402071 720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:18.402189 720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "39218fa275234524850eed75ef7b98fd"
format_stamp: "Formatted at 2026-08-09 19:34:18 on dist-test-slave-x4qh"
server_key: "22004e2d9cf737b79768093a80e43469"
server_key_iv: "d8f86aa0cceb3fca58708c75e92f2417"
server_key_version: "encryptionkey@0"
I20260809 19:34:18.402258 720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:18.423269 720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:18.425078 720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:18.425251 720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:18.429288 720 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32821
I20260809 19:34:18.429328 827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32821 every 8 connection(s)
I20260809 19:34:18.429718 720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:18.430534 831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:18.432673 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: Bootstrap starting.
I20260809 19:34:18.432679 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 720
I20260809 19:34:18.432809 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:18.433010 31975 external_mini_cluster.cc:1428] Setting key 082a6407b6dd1d9dbd422310aace1e43
I20260809 19:34:18.433398 831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:18.433753 831 log.cc:824] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:18.434506 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: No bootstrap required, opened a new log
I20260809 19:34:18.436475 831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 19:34:18.436622 831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:18.436664 831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39218fa275234524850eed75ef7b98fd, State: Initialized, Role: FOLLOWER
I20260809 19:34:18.436777 831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 19:34:18.436866 831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:18.436949 831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:18.436996 831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:18.437633 831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 19:34:18.437747 831 leader_election.cc:302] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39218fa275234524850eed75ef7b98fd; no voters:
I20260809 19:34:18.437891 831 leader_election.cc:288] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:18.437930 839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:18.438054 839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [term 1 LEADER]: Becoming Leader. State: Replica: 39218fa275234524850eed75ef7b98fd, State: Running, Role: LEADER
I20260809 19:34:18.438170 831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:18.438201 839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 19:34:18.438329 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:18.433484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40277 (local address 127.0.0.1:32821)
0809 19:34:18.433659 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:18.433666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:18.433693 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:34:18.434134 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:18.434140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:18.434182 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:18.434292 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:18.434489 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:18.435309 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:18.436416 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:18.436840 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:18.436975 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:18.437572 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:18.437575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:18.437576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:18.437754 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:18.437790 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:18.437802 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:18.437855 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:18.437923 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 19:34:18.437980 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 19:34:18.438547 839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39218fa275234524850eed75ef7b98fd. Latest consensus state: current_term: 1 leader_uuid: "39218fa275234524850eed75ef7b98fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260809 19:34:18.438629 842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39218fa275234524850eed75ef7b98fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39218fa275234524850eed75ef7b98fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260809 19:34:18.439038 839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:18.439050 842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:18.439103 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 19:34:18.439888 859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:18.439949 859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:18.440004 853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:18.440217 853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:18.441634 853 catalog_manager.cc:1347] Generated new cluster ID: 38ab657e19774e72a70f5427d7f8d8d7
I20260809 19:34:18.441681 853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:18.443356 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:18.440538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49659 (local address 127.0.0.1:32821)
0809 19:34:18.440569 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:18.440571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:18.440601 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:34:18.440657 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:18.440658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:18.440666 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:18.440716 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:18.440819 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:18.441358 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:18.442233 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:18.442485 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:18.442547 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:18.443161 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:18.443162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:18.443162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:18.443166 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:18.443178 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:18.443181 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:18.443230 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:18.443277 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 19:34:18.443294 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:34:18.457954 853 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:18.458617 853 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:18.467082 853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39218fa275234524850eed75ef7b98fd: Generated new TSK 0
I20260809 19:34:18.467234 853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:18.477025 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 720
2026-08-09T19:34:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:18 dist-test-slave-x4qh krb5kdc[876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:18 dist-test-slave-x4qh krb5kdc[876](info): set up 2 sockets
Aug 09 19:34:18 dist-test-slave-x4qh krb5kdc[876](info): commencing operation
krb5kdc: starting...
W20260809 19:34:19.680548 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:19 dist-test-slave-x4qh krb5kdc[876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:19.789857 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:19.866261 895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:19.866504 895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:19.866544 895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:19.867722 895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:19.867779 895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:19.867801 895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:19.867848 895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:19.867875 895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:19.869256 895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:19.869535 895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:19.869747 895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:19.874635 907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:19.874603 903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:19.874622 904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:19.875447 895 server_base.cc:1034] running on GCE node
I20260809 19:34:19.875708 895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:19.875982 895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:19.877161 895 hybrid_clock.cc:648] HybridClock initialized: now 1786304059877142 us; error 37 us; skew 500 ppm
Aug 09 19:34:19 dist-test-slave-x4qh krb5kdc[876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304059, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:19.879528 895 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:19.879966 895 webserver.cc:460] Webserver started at http://127.0.0.1:36175/ using document root <none> and password file <none>
I20260809 19:34:19.880144 895 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:19.880180 895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:19.880259 895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:19.881139 895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd"
format_stamp: "Formatted at 2026-08-09 19:34:19 on dist-test-slave-x4qh"
server_key: "92ec0493823b15a46aee0ca5221c4b47"
server_key_iv: "857a3c89b34387da34850315b604873b"
server_key_version: "encryptionkey@0"
I20260809 19:34:19.881418 895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd"
format_stamp: "Formatted at 2026-08-09 19:34:19 on dist-test-slave-x4qh"
server_key: "92ec0493823b15a46aee0ca5221c4b47"
server_key_iv: "857a3c89b34387da34850315b604873b"
server_key_version: "encryptionkey@0"
I20260809 19:34:19.882596 895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:19.883275 921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:19.883446 895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:19.883534 895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd"
format_stamp: "Formatted at 2026-08-09 19:34:19 on dist-test-slave-x4qh"
server_key: "92ec0493823b15a46aee0ca5221c4b47"
server_key_iv: "857a3c89b34387da34850315b604873b"
server_key_version: "encryptionkey@0"
I20260809 19:34:19.883587 895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:19.886700 895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:19.888490 895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:19.888592 895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:19.892261 895 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36433
I20260809 19:34:19.892315 1005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36433 every 8 connection(s)
I20260809 19:34:19.892524 895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:19.893313 1009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:19.894039 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 895
I20260809 19:34:19.894122 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:19.894320 31975 external_mini_cluster.cc:1428] Setting key b8c62eb9a8113f8e40c4268f0836616d
I20260809 19:34:19.896337 1009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd: Bootstrap starting.
I20260809 19:34:19.896755 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:19.894778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35283 (local address 127.0.0.1:36433)
0809 19:34:19.894935 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:19.894943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:19.894971 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:34:19.895512 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:19.895520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:19.895554 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:19.895691 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:19.896217 (+ 526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:19.896221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:19.896222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:19.896226 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:19.896263 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:19.896276 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:19.896358 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:19.896424 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:34:19.896564 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260809 19:34:19.897059 1009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:19.897380 1009 log.cc:824] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:19.897796 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:19.898038 1009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd: No bootstrap required, opened a new log
I20260809 19:34:19.900302 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:19.899349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55169 (local address 127.0.0.1:36433)
0809 19:34:19.899425 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:19.899427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:19.899493 (+ 66us) server_negotiation.cc:407] Connection header received
0809 19:34:19.899550 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:19.899551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:19.899554 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:19.899614 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:19.900111 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:19.900113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:19.900113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:19.900117 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:19.900130 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:19.900135 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:19.900211 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:19.900242 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:34:19.900252 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:19.900492 1009 raft_consensus.cc:348] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 19:34:19.900653 1009 raft_consensus.cc:374] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:19.900696 1009 raft_consensus.cc:729] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce4abec60c24ee18bb7fcb9ee4c39bd, State: Initialized, Role: FOLLOWER
I20260809 19:34:19.900811 1009 consensus_queue.cc:260] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 19:34:19.900898 1009 raft_consensus.cc:388] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:19.900955 1009 raft_consensus.cc:482] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:19.901003 1009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:19.901503 1009 raft_consensus.cc:504] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 19:34:19.901609 1009 leader_election.cc:302] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bce4abec60c24ee18bb7fcb9ee4c39bd; no voters:
I20260809 19:34:19.901794 1009 leader_election.cc:288] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:19.901823 1023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:19.901964 1023 raft_consensus.cc:686] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [term 1 LEADER]: Becoming Leader. State: Replica: bce4abec60c24ee18bb7fcb9ee4c39bd, State: Running, Role: LEADER
I20260809 19:34:19.902040 1009 sys_catalog.cc:564] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:19.902056 1023 consensus_queue.cc:237] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 19:34:19.902364 1023 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader bce4abec60c24ee18bb7fcb9ee4c39bd. Latest consensus state: current_term: 1 leader_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260809 19:34:19.902375 1026 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce4abec60c24ee18bb7fcb9ee4c39bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260809 19:34:19.902503 1023 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:19.902535 1026 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:19.902974 1032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:19.903533 1032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:19.905061 1032 catalog_manager.cc:1347] Generated new cluster ID: 3b92fd61259949f58242f1e4e70195a7
I20260809 19:34:19.905129 1032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:19.925910 1032 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:19.926065 1032 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:19.931798 1032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bce4abec60c24ee18bb7fcb9ee4c39bd: Generated new TSK 0
I20260809 19:34:19.931988 1032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:19.935493 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 895
2026-08-09T19:34:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:19 dist-test-slave-x4qh krb5kdc[1062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:19 dist-test-slave-x4qh krb5kdc[1062](info): set up 2 sockets
Aug 09 19:34:19 dist-test-slave-x4qh krb5kdc[1062](info): commencing operation
krb5kdc: starting...
W20260809 19:34:21.111572 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:21.219054 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:21.293682 1088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:21.293874 1088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:21.293896 1088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:21.294991 1088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:21.295030 1088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:21.295044 1088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:21.295092 1088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:21.295110 1088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:21.296523 1088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:21.296761 1088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:21.296988 1088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:21.301642 1098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:21.301651 1099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:21.301658 1104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:21.301934 1088 server_base.cc:1034] running on GCE node
I20260809 19:34:21.302150 1088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:21.302477 1088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:21.303637 1088 hybrid_clock.cc:648] HybridClock initialized: now 1786304061303610 us; error 37 us; skew 500 ppm
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304061, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:21.309993 1088 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:21.310384 1088 webserver.cc:460] Webserver started at http://127.0.0.1:44301/ using document root <none> and password file <none>
I20260809 19:34:21.310596 1088 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:21.310635 1088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:21.310798 1088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:21.311878 1088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "fbbcb0639b79497093eaf759892fba4d"
format_stamp: "Formatted at 2026-08-09 19:34:21 on dist-test-slave-x4qh"
server_key: "ad9da6b213714d7cd49477f4eda224eb"
server_key_iv: "888af05543981ced935e5b6c14f0e812"
server_key_version: "encryptionkey@0"
I20260809 19:34:21.312217 1088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "fbbcb0639b79497093eaf759892fba4d"
format_stamp: "Formatted at 2026-08-09 19:34:21 on dist-test-slave-x4qh"
server_key: "ad9da6b213714d7cd49477f4eda224eb"
server_key_iv: "888af05543981ced935e5b6c14f0e812"
server_key_version: "encryptionkey@0"
I20260809 19:34:21.313519 1088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:21.314180 1118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:21.314354 1088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:21.314459 1088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "fbbcb0639b79497093eaf759892fba4d"
format_stamp: "Formatted at 2026-08-09 19:34:21 on dist-test-slave-x4qh"
server_key: "ad9da6b213714d7cd49477f4eda224eb"
server_key_iv: "888af05543981ced935e5b6c14f0e812"
server_key_version: "encryptionkey@0"
I20260809 19:34:21.314530 1088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:21.327265 1088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:21.329308 1088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:21.329479 1088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:21.333287 1088 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45361
I20260809 19:34:21.333338 1205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45361 every 8 connection(s)
I20260809 19:34:21.333670 1088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:21.334465 1208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:21.336606 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d: Bootstrap starting.
I20260809 19:34:21.337216 1208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:21.337513 1208 log.cc:824] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:21.338181 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d: No bootstrap required, opened a new log
I20260809 19:34:21.339692 1208 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } }
I20260809 19:34:21.339830 1208 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:21.339879 1208 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbbcb0639b79497093eaf759892fba4d, State: Initialized, Role: FOLLOWER
I20260809 19:34:21.340045 1208 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } }
I20260809 19:34:21.340103 1208 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:21.340123 1208 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:21.340186 1208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:21.340695 1208 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } }
I20260809 19:34:21.340808 1208 leader_election.cc:302] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbbcb0639b79497093eaf759892fba4d; no voters:
I20260809 19:34:21.340994 1208 leader_election.cc:288] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:21.341020 1215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:21.341189 1215 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [term 1 LEADER]: Becoming Leader. State: Replica: fbbcb0639b79497093eaf759892fba4d, State: Running, Role: LEADER
I20260809 19:34:21.341238 1208 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:21.341331 1215 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } }
I20260809 19:34:21.341665 1217 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbbcb0639b79497093eaf759892fba4d. Latest consensus state: current_term: 1 leader_uuid: "fbbcb0639b79497093eaf759892fba4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } } }
I20260809 19:34:21.341795 1216 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbbcb0639b79497093eaf759892fba4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbcb0639b79497093eaf759892fba4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45361 } } }
I20260809 19:34:21.341805 1217 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:21.341898 1216 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:21.342394 1229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:21.342859 1229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:21.343072 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1088
I20260809 19:34:21.343173 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:21.343364 31975 external_mini_cluster.cc:1428] Setting key 87b78c98395b6756febe5ddec7880ec1
I20260809 19:34:21.344326 1229 catalog_manager.cc:1347] Generated new cluster ID: ffb3129d6e4147d890286ce9ebd68353
I20260809 19:34:21.344388 1229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:21.350695 1229 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:21.351449 1229 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:34:21.360783 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:21.343842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:46739 (local address 127.0.0.1:45361)
0809 19:34:21.344167 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:21.344175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:21.344203 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:34:21.350265 (+ 6062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:21.350273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:21.350331 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:21.350485 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:21.351289 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:21.352577 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:21.353571 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:21.354200 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:21.354306 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:21.356954 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:21.356958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:21.356958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:21.356969 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:21.358842 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:21.359305 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:21.359308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:21.359309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:21.359433 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:21.359689 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:21.359690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:21.359690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:21.359941 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:21.360393 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:21.360470 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:34:21.360561 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":78,"threads_started":1}
I20260809 19:34:21.361673 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:21.365327 1229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbbcb0639b79497093eaf759892fba4d: Generated new TSK 0
I20260809 19:34:21.365463 1229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:21.369678 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:21.363060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:49961 (local address 127.0.0.1:45361)
0809 19:34:21.363092 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:21.363094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:21.363197 (+ 103us) server_negotiation.cc:407] Connection header received
0809 19:34:21.363223 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:21.363223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:21.363232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:21.363303 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:21.363845 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:21.364655 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:21.365605 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:21.365917 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:21.365984 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:21.367037 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:21.367038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:21.367039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:21.367046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:21.368608 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:21.368948 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:21.368949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:21.368949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:21.369007 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:21.369221 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:21.369222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:21.369222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:21.369340 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:21.369444 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:21.369534 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 19:34:21.369597 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:34:21.371764 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1088
W20260809 19:34:21.388202 1056 connection.cc:535] client connection to 127.0.0.1:45361 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45361): Connection reset by peer (error 104)
2026-08-09T19:34:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1252](info): set up 2 sockets
Aug 09 19:34:21 dist-test-slave-x4qh krb5kdc[1252](info): commencing operation
krb5kdc: starting...
W20260809 19:34:22.556834 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:22 dist-test-slave-x4qh krb5kdc[1252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:22.665890 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:22.743216 1281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:22.743428 1281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:22.743453 1281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:22.744591 1281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:22.744637 1281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:22.744652 1281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:22.744663 1281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:22.744715 1281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:22.746158 1281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:22.746400 1281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:22.746623 1281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:22.751139 1289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:22.751137 1294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:22.751116 1291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:22.751168 1281 server_base.cc:1034] running on GCE node
I20260809 19:34:22.751691 1281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:22.751969 1281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:22.753127 1281 hybrid_clock.cc:648] HybridClock initialized: now 1786304062753114 us; error 43 us; skew 500 ppm
Aug 09 19:34:22 dist-test-slave-x4qh krb5kdc[1252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:22.755582 1281 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:22.756001 1281 webserver.cc:460] Webserver started at http://127.0.0.1:43647/ using document root <none> and password file <none>
I20260809 19:34:22.756240 1281 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:22.756313 1281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:22.756459 1281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:22.757418 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "0c53aa365ece481f881ecea1560323fa"
format_stamp: "Formatted at 2026-08-09 19:34:22 on dist-test-slave-x4qh"
server_key: "a7e809fc60085ea1d71cebb5e9f0bf24"
server_key_iv: "6604bda754e1dfab14c4f3429c3741c2"
server_key_version: "encryptionkey@0"
I20260809 19:34:22.757768 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "0c53aa365ece481f881ecea1560323fa"
format_stamp: "Formatted at 2026-08-09 19:34:22 on dist-test-slave-x4qh"
server_key: "a7e809fc60085ea1d71cebb5e9f0bf24"
server_key_iv: "6604bda754e1dfab14c4f3429c3741c2"
server_key_version: "encryptionkey@0"
I20260809 19:34:22.759034 1281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:34:22.759755 1305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:22.759915 1281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:22.760005 1281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "0c53aa365ece481f881ecea1560323fa"
format_stamp: "Formatted at 2026-08-09 19:34:22 on dist-test-slave-x4qh"
server_key: "a7e809fc60085ea1d71cebb5e9f0bf24"
server_key_iv: "6604bda754e1dfab14c4f3429c3741c2"
server_key_version: "encryptionkey@0"
I20260809 19:34:22.760080 1281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:22.772620 1281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:22.774269 1281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:22.774449 1281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:22.778281 1281 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42595
I20260809 19:34:22.778322 1392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42595 every 8 connection(s)
I20260809 19:34:22.778637 1281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:22.779556 1393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:22.780733 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1281
I20260809 19:34:22.780869 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:22.781085 31975 external_mini_cluster.cc:1428] Setting key 8dc223d64a22748bfd36c19fc3da950e
I20260809 19:34:22.782202 1393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: Bootstrap starting.
I20260809 19:34:22.782934 1393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:22.783264 1393 log.cc:824] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:22.784014 1393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: No bootstrap required, opened a new log
I20260809 19:34:22.786087 1393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } }
I20260809 19:34:22.786233 1393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:22.786274 1393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c53aa365ece481f881ecea1560323fa, State: Initialized, Role: FOLLOWER
I20260809 19:34:22.786396 1393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } }
I20260809 19:34:22.786489 1393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:22.786521 1393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:22.786569 1393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:22.786935 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:22.781599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:47901 (local address 127.0.0.1:42595)
0809 19:34:22.781808 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:22.781817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:22.781858 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:34:22.782391 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:22.782399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:22.782456 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:22.782615 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:22.782787 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:22.784165 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:22.784927 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:22.785546 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:22.785677 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:22.786277 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:22.786289 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:22.786290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:22.786484 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:22.786521 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:22.786537 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:22.786624 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:22.786712 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 19:34:22.786739 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260809 19:34:22.787220 1393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } }
I20260809 19:34:22.787307 1393 leader_election.cc:302] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c53aa365ece481f881ecea1560323fa; no voters:
I20260809 19:34:22.787506 1393 leader_election.cc:288] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:22.787568 1400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:22.787731 1400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [term 1 LEADER]: Becoming Leader. State: Replica: 0c53aa365ece481f881ecea1560323fa, State: Running, Role: LEADER
I20260809 19:34:22.787815 1393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:22.787845 1400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } }
I20260809 19:34:22.788036 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:22.788110 1400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c53aa365ece481f881ecea1560323fa. Latest consensus state: current_term: 1 leader_uuid: "0c53aa365ece481f881ecea1560323fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } } }
I20260809 19:34:22.788116 1403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c53aa365ece481f881ecea1560323fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c53aa365ece481f881ecea1560323fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42595 } } }
I20260809 19:34:22.788187 1400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:22.788188 1403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:22.789570 1424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:22.789672 1424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:22.789813 1412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:22.790055 1412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:22.791543 1412 catalog_manager.cc:1347] Generated new cluster ID: e54931f20a6e4deb882b9332f295c170
I20260809 19:34:22.791592 1412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:22.792732 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:22.789656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:49139 (local address 127.0.0.1:42595)
0809 19:34:22.789693 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:22.789695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:22.789757 (+ 62us) server_negotiation.cc:407] Connection header received
0809 19:34:22.789844 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:22.789845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:22.789852 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:22.789947 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:22.790086 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:22.790564 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:22.791549 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:22.791807 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:22.791885 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:22.792524 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:22.792525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:22.792526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:22.792530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:22.792541 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:22.792545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:22.792597 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:22.792655 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 19:34:22.792662 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:22.810195 1412 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:22.810684 1412 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:22.813385 1412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c53aa365ece481f881ecea1560323fa: Generated new TSK 0
I20260809 19:34:22.813561 1412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:22.820356 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1281
W20260809 19:34:22.834852 1250 connection.cc:535] client connection to 127.0.0.1:42595 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42595): Connection reset by peer (error 104)
2026-08-09T19:34:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:22 dist-test-slave-x4qh krb5kdc[1440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:22 dist-test-slave-x4qh krb5kdc[1440](info): set up 2 sockets
Aug 09 19:34:22 dist-test-slave-x4qh krb5kdc[1440](info): commencing operation
krb5kdc: starting...
W20260809 19:34:23.991931 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:24 dist-test-slave-x4qh krb5kdc[1440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:24.101763 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:24.178406 1459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:24.178618 1459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:24.178670 1459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:24.179886 1459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:24.179951 1459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:24.179975 1459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:24.179996 1459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:24.180023 1459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:24.181494 1459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:24.181770 1459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:24.181998 1459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:24.186717 1466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:24.186733 1470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:24.186722 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:24.187049 1459 server_base.cc:1034] running on GCE node
I20260809 19:34:24.187251 1459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:24.187542 1459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:24.188706 1459 hybrid_clock.cc:648] HybridClock initialized: now 1786304064188694 us; error 45 us; skew 500 ppm
Aug 09 19:34:24 dist-test-slave-x4qh krb5kdc[1440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:24.191433 1459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:24.191809 1459 webserver.cc:460] Webserver started at http://127.0.0.1:41701/ using document root <none> and password file <none>
I20260809 19:34:24.192041 1459 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:24.192108 1459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:24.192270 1459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:24.193245 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "f36f200c2ef94c3c9b4e962e68bf499a"
format_stamp: "Formatted at 2026-08-09 19:34:24 on dist-test-slave-x4qh"
server_key: "5934b86be44d38b33e65b819a2bc0a2e"
server_key_iv: "69e93f199bf1c106771c891d8db15632"
server_key_version: "encryptionkey@0"
I20260809 19:34:24.193621 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "f36f200c2ef94c3c9b4e962e68bf499a"
format_stamp: "Formatted at 2026-08-09 19:34:24 on dist-test-slave-x4qh"
server_key: "5934b86be44d38b33e65b819a2bc0a2e"
server_key_iv: "69e93f199bf1c106771c891d8db15632"
server_key_version: "encryptionkey@0"
I20260809 19:34:24.194883 1459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:24.195700 1488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:24.195883 1459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:24.195982 1459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "f36f200c2ef94c3c9b4e962e68bf499a"
format_stamp: "Formatted at 2026-08-09 19:34:24 on dist-test-slave-x4qh"
server_key: "5934b86be44d38b33e65b819a2bc0a2e"
server_key_iv: "69e93f199bf1c106771c891d8db15632"
server_key_version: "encryptionkey@0"
I20260809 19:34:24.196081 1459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:24.199126 1459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:24.200836 1459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:24.200987 1459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:24.205598 1459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39595
I20260809 19:34:24.205649 1562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39595 every 8 connection(s)
I20260809 19:34:24.205968 1459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:24.206784 1563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:24.209059 1563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a: Bootstrap starting.
I20260809 19:34:24.209640 1563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:24.209937 1563 log.cc:824] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:24.210593 1563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a: No bootstrap required, opened a new log
I20260809 19:34:24.212218 1563 raft_consensus.cc:348] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } }
I20260809 19:34:24.212354 1563 raft_consensus.cc:374] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:24.212399 1563 raft_consensus.cc:729] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36f200c2ef94c3c9b4e962e68bf499a, State: Initialized, Role: FOLLOWER
I20260809 19:34:24.212576 1563 consensus_queue.cc:260] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } }
I20260809 19:34:24.212656 1563 raft_consensus.cc:388] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:24.212702 1563 raft_consensus.cc:482] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:24.212752 1563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:24.213236 1563 raft_consensus.cc:504] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } }
I20260809 19:34:24.213347 1563 leader_election.cc:302] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f36f200c2ef94c3c9b4e962e68bf499a; no voters:
I20260809 19:34:24.213570 1563 leader_election.cc:288] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:24.213634 1570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:24.213811 1563 sys_catalog.cc:564] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:24.213824 1570 raft_consensus.cc:686] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [term 1 LEADER]: Becoming Leader. State: Replica: f36f200c2ef94c3c9b4e962e68bf499a, State: Running, Role: LEADER
I20260809 19:34:24.213948 1570 consensus_queue.cc:237] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } }
I20260809 19:34:24.214354 1571 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } } }
I20260809 19:34:24.214524 1571 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:24.214387 1572 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f36f200c2ef94c3c9b4e962e68bf499a. Latest consensus state: current_term: 1 leader_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36f200c2ef94c3c9b4e962e68bf499a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39595 } } }
I20260809 19:34:24.214918 1572 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:24.215050 1580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:24.215602 1580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:24.216045 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1459
I20260809 19:34:24.216173 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:24.216394 31975 external_mini_cluster.cc:1428] Setting key 731e9241ce671299144f923388962004
I20260809 19:34:24.217665 1580 catalog_manager.cc:1347] Generated new cluster ID: 9d75d8883ac441039bcc5074648692e2
I20260809 19:34:24.219348 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:24.217206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:53375 (local address 127.0.0.1:39595)
0809 19:34:24.217839 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:24.217848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:24.217881 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:34:24.217996 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:24.218003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:24.218034 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:24.218171 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:24.218863 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:24.218867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:24.218867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:24.218871 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:24.218922 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:24.218936 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:24.218991 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:24.219139 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 19:34:24.219178 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":102,"threads_started":1}
I20260809 19:34:24.219434 1580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:24.228219 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:24.230719 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:24.229629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:46975 (local address 127.0.0.1:39595)
0809 19:34:24.229665 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:24.229667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:24.229783 (+ 116us) server_negotiation.cc:407] Connection header received
0809 19:34:24.229870 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:24.229871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:24.229875 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:24.229901 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:24.230502 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:24.230503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:24.230504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:24.230507 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:24.230520 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:24.230525 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:24.230552 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:24.230646 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:34:24.230660 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:24.233172 1580 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:24.233295 1580 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:24.235709 1580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f36f200c2ef94c3c9b4e962e68bf499a: Generated new TSK 0
I20260809 19:34:24.235868 1580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:24.237655 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1459
2026-08-09T19:34:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:24 dist-test-slave-x4qh krb5kdc[1614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:24 dist-test-slave-x4qh krb5kdc[1614](info): set up 2 sockets
Aug 09 19:34:24 dist-test-slave-x4qh krb5kdc[1614](info): commencing operation
krb5kdc: starting...
W20260809 19:34:25.436444 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:25.551644 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:25.628701 1635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:25.628890 1635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:25.628911 1635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:25.630008 1635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:25.630051 1635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:25.630064 1635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:25.630115 1635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:25.630134 1635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:25.631609 1635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:25.631879 1635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:25.632110 1635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:25.636898 1647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:25.636940 1646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:25.636958 1649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:25.637131 1635 server_base.cc:1034] running on GCE node
I20260809 19:34:25.637352 1635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:25.637631 1635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:25.638806 1635 hybrid_clock.cc:648] HybridClock initialized: now 1786304065638790 us; error 28 us; skew 500 ppm
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304065, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:25.641319 1635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:25.641737 1635 webserver.cc:460] Webserver started at http://127.0.0.1:43161/ using document root <none> and password file <none>
I20260809 19:34:25.641986 1635 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:25.642055 1635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:25.642200 1635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:25.643165 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "2c62b4f990f84608aa5861c41da2e000"
format_stamp: "Formatted at 2026-08-09 19:34:25 on dist-test-slave-x4qh"
server_key: "7d87c090e51b152341bd7acd952b5051"
server_key_iv: "696c5fa14303d4a0fdba567f59c27223"
server_key_version: "encryptionkey@0"
I20260809 19:34:25.643579 1635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "2c62b4f990f84608aa5861c41da2e000"
format_stamp: "Formatted at 2026-08-09 19:34:25 on dist-test-slave-x4qh"
server_key: "7d87c090e51b152341bd7acd952b5051"
server_key_iv: "696c5fa14303d4a0fdba567f59c27223"
server_key_version: "encryptionkey@0"
I20260809 19:34:25.644912 1635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:34:25.645705 1660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:25.645884 1635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:25.645994 1635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "2c62b4f990f84608aa5861c41da2e000"
format_stamp: "Formatted at 2026-08-09 19:34:25 on dist-test-slave-x4qh"
server_key: "7d87c090e51b152341bd7acd952b5051"
server_key_iv: "696c5fa14303d4a0fdba567f59c27223"
server_key_version: "encryptionkey@0"
I20260809 19:34:25.646070 1635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:25.655092 1635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:25.656908 1635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:25.657054 1635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:25.660657 1635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41033
I20260809 19:34:25.660725 1728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41033 every 8 connection(s)
I20260809 19:34:25.661031 1635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:25.661886 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:25.664264 1729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: Bootstrap starting.
I20260809 19:34:25.664930 1729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:25.665225 1729 log.cc:824] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:25.665781 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1635
I20260809 19:34:25.665908 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:25.666039 1729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: No bootstrap required, opened a new log
I20260809 19:34:25.666142 31975 external_mini_cluster.cc:1428] Setting key 57adeabacf313f096b9750e7bf017a7b
I20260809 19:34:25.668485 1729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260809 19:34:25.668632 1729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:25.668678 1729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c62b4f990f84608aa5861c41da2e000, State: Initialized, Role: FOLLOWER
I20260809 19:34:25.668797 1729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260809 19:34:25.668891 1729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:25.668938 1729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:25.668985 1729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:25.669533 1729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260809 19:34:25.669647 1729 leader_election.cc:302] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c62b4f990f84608aa5861c41da2e000; no voters:
I20260809 19:34:25.669808 1729 leader_election.cc:288] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:25.669866 1739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:25.670039 1729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:25.670043 1739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [term 1 LEADER]: Becoming Leader. State: Replica: 2c62b4f990f84608aa5861c41da2e000, State: Running, Role: LEADER
I20260809 19:34:25.670194 1739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } }
I20260809 19:34:25.670625 1743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c62b4f990f84608aa5861c41da2e000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
I20260809 19:34:25.670642 1744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c62b4f990f84608aa5861c41da2e000. Latest consensus state: current_term: 1 leader_uuid: "2c62b4f990f84608aa5861c41da2e000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c62b4f990f84608aa5861c41da2e000" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41033 } } }
I20260809 19:34:25.670805 1743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:25.670864 1744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000 [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:25.671931 1766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:25.672099 1766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:25.672199 1756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:25.672430 1756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:34:25.673761 1756 catalog_manager.cc:1347] Generated new cluster ID: 0127d3aacb8c4c0385d79b702899c8d2
I20260809 19:34:25.673823 1756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:25.678467 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:25.666606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:48793 (local address 127.0.0.1:41033)
0809 19:34:25.666860 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:25.666868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:25.666890 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:34:25.667435 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:25.667442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:25.667490 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:25.667595 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:25.668393 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:25.669620 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:25.670698 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:25.671399 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:25.671506 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:25.674277 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:25.674281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:25.674282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:25.674294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:25.676178 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:25.676658 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:25.676661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:25.676661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:25.676765 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:25.677026 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:25.677027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:25.677027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:25.677320 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:25.677460 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:25.677965 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 19:34:25.678115 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260809 19:34:25.679461 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:25.688012 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:25.680974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:54475 (local address 127.0.0.1:41033)
0809 19:34:25.681006 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:25.681008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:25.681062 (+ 54us) server_negotiation.cc:407] Connection header received
0809 19:34:25.681113 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:25.681113 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:25.681122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:25.681208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:25.681766 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:25.682484 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:25.683506 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:25.683836 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:25.683903 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:25.685155 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:25.685157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:25.685157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:25.685164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:25.686596 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:25.686977 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:25.686978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:25.686978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:25.687033 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:25.687239 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:25.687240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:25.687241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:25.687353 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:25.687804 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:25.687854 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:34:25.687911 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:25.696753 1756 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:25.697252 1756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:25.702880 1756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c62b4f990f84608aa5861c41da2e000: Generated new TSK 0
I20260809 19:34:25.703047 1756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:25.709066 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1635
W20260809 19:34:25.716148 1612 connection.cc:535] client connection to 127.0.0.1:41033 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41033): Connection reset by peer (error 104)
2026-08-09T19:34:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1784](info): set up 2 sockets
Aug 09 19:34:25 dist-test-slave-x4qh krb5kdc[1784](info): commencing operation
krb5kdc: starting...
W20260809 19:34:26.878686 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:26 dist-test-slave-x4qh krb5kdc[1784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:26.988245 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:27.062728 1801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:27.062908 1801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:27.062932 1801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:27.064106 1801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:27.064149 1801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:27.064164 1801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:27.064178 1801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:27.064231 1801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:27.065639 1801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:27.065869 1801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:27.066084 1801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:27.070667 1812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:27.070722 1801 server_base.cc:1034] running on GCE node
W20260809 19:34:27.070727 1815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:27.070674 1817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:27.071100 1801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:27.071414 1801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:27.072566 1801 hybrid_clock.cc:648] HybridClock initialized: now 1786304067072553 us; error 31 us; skew 500 ppm
Aug 09 19:34:27 dist-test-slave-x4qh krb5kdc[1784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304067, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:27.079936 1801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:27.080358 1801 webserver.cc:460] Webserver started at http://127.0.0.1:44853/ using document root <none> and password file <none>
I20260809 19:34:27.080582 1801 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:27.080626 1801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:27.080767 1801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:27.081717 1801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "73e5aefb47de463f8a6c4bb1bb4997b1"
format_stamp: "Formatted at 2026-08-09 19:34:27 on dist-test-slave-x4qh"
server_key: "b11ede857b55079791c980c3783d035b"
server_key_iv: "2c284e421e61a7ec18f0f16cce2d286a"
server_key_version: "encryptionkey@0"
I20260809 19:34:27.082041 1801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "73e5aefb47de463f8a6c4bb1bb4997b1"
format_stamp: "Formatted at 2026-08-09 19:34:27 on dist-test-slave-x4qh"
server_key: "b11ede857b55079791c980c3783d035b"
server_key_iv: "2c284e421e61a7ec18f0f16cce2d286a"
server_key_version: "encryptionkey@0"
I20260809 19:34:27.083304 1801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:27.084080 1831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:27.084259 1801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:27.084374 1801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "73e5aefb47de463f8a6c4bb1bb4997b1"
format_stamp: "Formatted at 2026-08-09 19:34:27 on dist-test-slave-x4qh"
server_key: "b11ede857b55079791c980c3783d035b"
server_key_iv: "2c284e421e61a7ec18f0f16cce2d286a"
server_key_version: "encryptionkey@0"
I20260809 19:34:27.084466 1801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:27.103159 1801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:27.105206 1801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:27.105309 1801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:27.108870 1801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36507
I20260809 19:34:27.108922 1916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36507 every 8 connection(s)
I20260809 19:34:27.109227 1801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:27.110029 1917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:27.112092 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1801
I20260809 19:34:27.112205 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:27.112283 1917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1: Bootstrap starting.
I20260809 19:34:27.112432 31975 external_mini_cluster.cc:1428] Setting key 9b34f4af517f2dbdbbe3aae952172971
I20260809 19:34:27.112962 1917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:27.113343 1917 log.cc:824] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:27.114081 1917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1: No bootstrap required, opened a new log
I20260809 19:34:27.115921 1917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } }
I20260809 19:34:27.116057 1917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:27.116101 1917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e5aefb47de463f8a6c4bb1bb4997b1, State: Initialized, Role: FOLLOWER
I20260809 19:34:27.116215 1917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } }
I20260809 19:34:27.116309 1917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:27.116351 1917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:27.116398 1917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:27.116882 1917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } }
I20260809 19:34:27.116990 1917 leader_election.cc:302] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73e5aefb47de463f8a6c4bb1bb4997b1; no voters:
I20260809 19:34:27.117125 1917 leader_election.cc:288] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:27.117151 1927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:27.117252 1927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [term 1 LEADER]: Becoming Leader. State: Replica: 73e5aefb47de463f8a6c4bb1bb4997b1, State: Running, Role: LEADER
I20260809 19:34:27.117347 1917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:27.117391 1927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } }
I20260809 19:34:27.117799 1927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73e5aefb47de463f8a6c4bb1bb4997b1. Latest consensus state: current_term: 1 leader_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } } }
I20260809 19:34:27.117940 1927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:27.117830 1930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e5aefb47de463f8a6c4bb1bb4997b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36507 } } }
I20260809 19:34:27.118072 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:27.113045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:33825 (local address 127.0.0.1:36507)
0809 19:34:27.113219 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:27.113228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:27.113270 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:34:27.113809 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:27.113818 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:27.113880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:27.114020 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:27.114244 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:27.115079 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:27.116002 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:27.116552 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:27.116668 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:27.117294 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:27.117298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:27.117298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:27.117482 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:27.117526 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:27.117546 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:27.117663 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:27.117737 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 19:34:27.117847 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260809 19:34:27.118132 1942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:27.118145 1930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:27.119134 1942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:27.119237 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:27.120708 1942 catalog_manager.cc:1347] Generated new cluster ID: 289b66beaf9449ddaaab558a3af1f543
I20260809 19:34:27.120769 1942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:27.126935 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:27.123535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:37049 (local address 127.0.0.1:36507)
0809 19:34:27.123590 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:27.123592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:27.123601 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:27.123620 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:27.123621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:27.123634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:27.123694 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:27.123846 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:27.124904 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:27.125776 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:27.126071 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:27.126140 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:27.126714 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:27.126715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:27.126715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:27.126719 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:27.126733 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:27.126737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:27.126799 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:27.126864 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:34:27.126874 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:34:27.133535 1942 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:27.134022 1942 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:27.138238 1942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73e5aefb47de463f8a6c4bb1bb4997b1: Generated new TSK 0
I20260809 19:34:27.138402 1942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:27.143252 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1801
W20260809 19:34:27.158239 1782 connection.cc:535] client connection to 127.0.0.1:36507 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36507): Connection reset by peer (error 104)
2026-08-09T19:34:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:27 dist-test-slave-x4qh krb5kdc[1964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:27 dist-test-slave-x4qh krb5kdc[1964](info): set up 2 sockets
Aug 09 19:34:27 dist-test-slave-x4qh krb5kdc[1964](info): commencing operation
krb5kdc: starting...
W20260809 19:34:28.319027 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:28 dist-test-slave-x4qh krb5kdc[1964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:34:28.434350 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:28.510094 1990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:28.510301 1990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:28.510337 1990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:28.511399 1990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:28.511461 1990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:28.511482 1990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:28.511502 1990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:28.511528 1990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:28.512840 1990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:28.513103 1990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:28.513329 1990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:28.517997 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:28.518011 2001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:28.518187 1990 server_base.cc:1034] running on GCE node
W20260809 19:34:28.518222 2004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:28.518488 1990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:28.518772 1990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:28.519942 1990 hybrid_clock.cc:648] HybridClock initialized: now 1786304068519917 us; error 37 us; skew 500 ppm
Aug 09 19:34:28 dist-test-slave-x4qh krb5kdc[1964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304068, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:28.522320 1990 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:34:28.522698 1990 webserver.cc:460] Webserver started at http://127.0.0.1:36689/ using document root <none> and password file <none>
I20260809 19:34:28.522894 1990 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:28.522934 1990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:28.523030 1990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:28.524036 1990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "a3d1f50376564a679a1fb45de398d745"
format_stamp: "Formatted at 2026-08-09 19:34:28 on dist-test-slave-x4qh"
server_key: "0435c5b110c5dd14aa5b7de967ada1e5"
server_key_iv: "870cfafe4d266aa83e5cffa38b55c065"
server_key_version: "encryptionkey@0"
I20260809 19:34:28.524364 1990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "a3d1f50376564a679a1fb45de398d745"
format_stamp: "Formatted at 2026-08-09 19:34:28 on dist-test-slave-x4qh"
server_key: "0435c5b110c5dd14aa5b7de967ada1e5"
server_key_iv: "870cfafe4d266aa83e5cffa38b55c065"
server_key_version: "encryptionkey@0"
I20260809 19:34:28.525555 1990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:28.526196 2011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:28.526381 1990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:28.526465 1990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "a3d1f50376564a679a1fb45de398d745"
format_stamp: "Formatted at 2026-08-09 19:34:28 on dist-test-slave-x4qh"
server_key: "0435c5b110c5dd14aa5b7de967ada1e5"
server_key_iv: "870cfafe4d266aa83e5cffa38b55c065"
server_key_version: "encryptionkey@0"
I20260809 19:34:28.526516 1990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:28.529541 1990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:28.531630 1990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:28.531733 1990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:28.535537 1990 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36155
I20260809 19:34:28.535612 2093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36155 every 8 connection(s)
I20260809 19:34:28.535899 1990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:28.536725 2095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:28.538194 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 1990
I20260809 19:34:28.538306 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:28.538486 31975 external_mini_cluster.cc:1428] Setting key 2e1fef9b3aeff73e807157c34d878bcf
I20260809 19:34:28.539227 2095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745: Bootstrap starting.
I20260809 19:34:28.539963 2095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:28.540298 2095 log.cc:824] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:28.541072 2095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745: No bootstrap required, opened a new log
I20260809 19:34:28.541505 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:28.538986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:47091 (local address 127.0.0.1:36155)
0809 19:34:28.539147 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:28.539155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:28.539583 (+ 428us) server_negotiation.cc:407] Connection header received
0809 19:34:28.540198 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:28.540208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:28.540242 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:28.540392 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:28.541027 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:28.541030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:28.541030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:28.541035 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:28.541076 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:28.541090 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:28.541182 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:28.541280 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:34:28.541319 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260809 19:34:28.542420 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:28.542994 2095 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260809 19:34:28.543119 2095 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:28.543140 2095 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d1f50376564a679a1fb45de398d745, State: Initialized, Role: FOLLOWER
I20260809 19:34:28.543264 2095 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260809 19:34:28.543407 2095 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:28.543503 2095 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:28.543562 2095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:28.544361 2095 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260809 19:34:28.544482 2095 leader_election.cc:302] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3d1f50376564a679a1fb45de398d745; no voters:
I20260809 19:34:28.544649 2095 leader_election.cc:288] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:28.544703 2109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:28.544801 2098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:28.543760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:34123 (local address 127.0.0.1:36155)
0809 19:34:28.543815 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:28.543816 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:28.543898 (+ 82us) server_negotiation.cc:407] Connection header received
0809 19:34:28.543961 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:28.543962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:28.543965 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:28.544008 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:28.544546 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:28.544547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:28.544547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:28.544553 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:28.544568 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:28.544572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:28.544628 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:28.544703 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 19:34:28.544717 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:34:28.544906 2109 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [term 1 LEADER]: Becoming Leader. State: Replica: a3d1f50376564a679a1fb45de398d745, State: Running, Role: LEADER
I20260809 19:34:28.545012 2109 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260809 19:34:28.545015 2095 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:28.545369 2112 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3d1f50376564a679a1fb45de398d745. Latest consensus state: current_term: 1 leader_uuid: "a3d1f50376564a679a1fb45de398d745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } } }
I20260809 19:34:28.545459 2112 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:28.545644 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3d1f50376564a679a1fb45de398d745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d1f50376564a679a1fb45de398d745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } } }
I20260809 19:34:28.545692 2115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:28.545781 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:28.546257 2115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:28.547876 2115 catalog_manager.cc:1347] Generated new cluster ID: 67544e7a6ebc474b80b75d09d3a63690
I20260809 19:34:28.547930 2115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:28.573987 2115 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:28.574152 2115 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:28.577124 2115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3d1f50376564a679a1fb45de398d745: Generated new TSK 0
I20260809 19:34:28.577288 2115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:28.579584 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 1990
2026-08-09T19:34:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:28 dist-test-slave-x4qh krb5kdc[2137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:28 dist-test-slave-x4qh krb5kdc[2137](info): set up 2 sockets
Aug 09 19:34:28 dist-test-slave-x4qh krb5kdc[2137](info): commencing operation
krb5kdc: starting...
W20260809 19:34:29.776639 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:29 dist-test-slave-x4qh krb5kdc[2137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:29.885881 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41259
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:41259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:29.961652 2158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:29.961874 2158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:29.961911 2158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:29.962961 2158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:29.963032 2158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:29.963054 2158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:29.963073 2158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:29.963100 2158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:29.964524 2158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:41259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:29.964797 2158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:29.965047 2158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:29.969655 2170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:29.969619 2171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:29.969784 2158 server_base.cc:1034] running on GCE node
W20260809 19:34:29.969632 2173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:29.970043 2158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:29.970336 2158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:29.971498 2158 hybrid_clock.cc:648] HybridClock initialized: now 1786304069971486 us; error 44 us; skew 500 ppm
Aug 09 19:34:29 dist-test-slave-x4qh krb5kdc[2137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304069, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:29.973834 2158 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:29.974274 2158 webserver.cc:460] Webserver started at http://127.31.57.254:33045/ using document root <none> and password file <none>
I20260809 19:34:29.974529 2158 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:29.974598 2158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:29.974745 2158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:29.975811 2158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "3f98dbfc2d0945bca401edd2fc10577b"
format_stamp: "Formatted at 2026-08-09 19:34:29 on dist-test-slave-x4qh"
server_key: "d775e0dacc5f7342318b9d61e5e71f81"
server_key_iv: "2098757429a1e5b63ea04c17a46bec8e"
server_key_version: "encryptionkey@0"
I20260809 19:34:29.976156 2158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "3f98dbfc2d0945bca401edd2fc10577b"
format_stamp: "Formatted at 2026-08-09 19:34:29 on dist-test-slave-x4qh"
server_key: "d775e0dacc5f7342318b9d61e5e71f81"
server_key_iv: "2098757429a1e5b63ea04c17a46bec8e"
server_key_version: "encryptionkey@0"
I20260809 19:34:29.977437 2158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:29.978195 2182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:29.978361 2158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:29.978469 2158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "3f98dbfc2d0945bca401edd2fc10577b"
format_stamp: "Formatted at 2026-08-09 19:34:29 on dist-test-slave-x4qh"
server_key: "d775e0dacc5f7342318b9d61e5e71f81"
server_key_iv: "2098757429a1e5b63ea04c17a46bec8e"
server_key_version: "encryptionkey@0"
I20260809 19:34:29.978545 2158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:29.995834 2158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:29.997359 2158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:29.997534 2158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:30.001132 2158 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:41259
I20260809 19:34:30.001159 2266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:41259 every 8 connection(s)
I20260809 19:34:30.001467 2158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:30.002287 2269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:30.004417 2269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b: Bootstrap starting.
I20260809 19:34:30.004948 2269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:30.005193 2269 log.cc:824] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:30.005817 2269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b: No bootstrap required, opened a new log
I20260809 19:34:30.007184 2269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } }
I20260809 19:34:30.007302 2269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:30.007321 2269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f98dbfc2d0945bca401edd2fc10577b, State: Initialized, Role: FOLLOWER
I20260809 19:34:30.007463 2269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } }
I20260809 19:34:30.007539 2269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:30.007560 2269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:30.007583 2269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:30.008296 2269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } }
I20260809 19:34:30.008378 2269 leader_election.cc:302] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f98dbfc2d0945bca401edd2fc10577b; no voters:
I20260809 19:34:30.008487 2269 leader_election.cc:288] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:30.008551 2273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:30.008704 2273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [term 1 LEADER]: Becoming Leader. State: Replica: 3f98dbfc2d0945bca401edd2fc10577b, State: Running, Role: LEADER
I20260809 19:34:30.008738 2269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:30.008790 2273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } }
I20260809 19:34:30.009140 2274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f98dbfc2d0945bca401edd2fc10577b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } } }
I20260809 19:34:30.009150 2275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f98dbfc2d0945bca401edd2fc10577b. Latest consensus state: current_term: 1 leader_uuid: "3f98dbfc2d0945bca401edd2fc10577b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f98dbfc2d0945bca401edd2fc10577b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 41259 } } }
I20260809 19:34:30.009243 2274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:30.009289 2275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:30.009696 2285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:30.010181 2285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:30.010515 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2158
I20260809 19:34:30.010639 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:30.010838 31975 external_mini_cluster.cc:1428] Setting key fd5fcaf0e67559681ba1b74bcfcd35ab
I20260809 19:34:30.012394 2285 catalog_manager.cc:1347] Generated new cluster ID: d2cdf14e019c4171b40d21e7639f2435
I20260809 19:34:30.012454 2285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:30 dist-test-slave-x4qh krb5kdc[2137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:30.022137 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:30.011281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45959 (local address 127.31.57.254:41259)
0809 19:34:30.011597 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:30.011605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:30.011629 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:34:30.011727 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:30.011733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:30.011779 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:30.011931 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:30.012696 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:30.014016 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:30.015118 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:30.015659 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:30.015745 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:30.018342 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:30.018346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:30.018346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:30.018357 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:30.020415 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:30.020852 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:30.020855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:30.020856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:30.020975 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:30.021207 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:30.021208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:30.021208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:30.021532 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:30.021690 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:30.021802 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:34:30.021908 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":109,"threads_started":1}
I20260809 19:34:30.023195 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:30.031481 2285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:30.032119 2285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:30.035403 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:30.027718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33885 (local address 127.31.57.254:41259)
0809 19:34:30.027765 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:30.027767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:30.027892 (+ 125us) server_negotiation.cc:407] Connection header received
0809 19:34:30.027947 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:30.027947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:30.027957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:30.028021 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:30.028565 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:30.029428 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:30.030408 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:30.031313 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:30.031451 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:30.032640 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:30.032642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:30.032642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:30.032649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:30.034134 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:30.034534 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:30.034535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:30.034536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:30.034604 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:30.034844 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:30.034845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:30.034845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:30.034967 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:30.035077 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:30.035234 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 19:34:30.035303 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:34:30.037070 2285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f98dbfc2d0945bca401edd2fc10577b: Generated new TSK 0
I20260809 19:34:30.037209 2285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:30.043259 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2158
W20260809 19:34:30.055820 2134 connection.cc:535] client connection to 127.31.57.254:41259 recv error: Network error: failed to read from TLS socket (remote: 127.31.57.254:41259): Connection reset by peer (error 104)
2026-08-09T19:34:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:30 dist-test-slave-x4qh krb5kdc[2319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:30 dist-test-slave-x4qh krb5kdc[2319](info): set up 2 sockets
Aug 09 19:34:30 dist-test-slave-x4qh krb5kdc[2319](info): commencing operation
krb5kdc: starting...
W20260809 19:34:31.211884 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:31 dist-test-slave-x4qh krb5kdc[2319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:31.319789 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34519
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:34519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:31.396589 2340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:31.396783 2340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:31.396816 2340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:31.397910 2340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:31.397984 2340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:31.398007 2340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:31.398032 2340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:31.398053 2340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:31.399456 2340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:34519
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:34519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:31.399693 2340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:31.399896 2340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:34:31.404484 2340 server_base.cc:1034] running on GCE node
W20260809 19:34:31.404464 2347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:31.404469 2348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:31.404452 2352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:31.404980 2340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:31.405254 2340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:31.406423 2340 hybrid_clock.cc:648] HybridClock initialized: now 1786304071406404 us; error 34 us; skew 500 ppm
Aug 09 19:34:31 dist-test-slave-x4qh krb5kdc[2319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304071, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:31.408864 2340 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:31.409308 2340 webserver.cc:460] Webserver started at http://127.31.57.254:33957/ using document root <none> and password file <none>
I20260809 19:34:31.409533 2340 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:31.409570 2340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:31.409650 2340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:31.410526 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "4a955a4d0e194deba3924625ab8b3754"
format_stamp: "Formatted at 2026-08-09 19:34:31 on dist-test-slave-x4qh"
server_key: "013fa66657066c887959cb828a54c7eb"
server_key_iv: "c034b6e2bfe88b185ea8516fedada3cf"
server_key_version: "encryptionkey@0"
I20260809 19:34:31.410794 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "4a955a4d0e194deba3924625ab8b3754"
format_stamp: "Formatted at 2026-08-09 19:34:31 on dist-test-slave-x4qh"
server_key: "013fa66657066c887959cb828a54c7eb"
server_key_iv: "c034b6e2bfe88b185ea8516fedada3cf"
server_key_version: "encryptionkey@0"
I20260809 19:34:31.412056 2340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:34:31.412714 2362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:31.412917 2340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:31.412997 2340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "4a955a4d0e194deba3924625ab8b3754"
format_stamp: "Formatted at 2026-08-09 19:34:31 on dist-test-slave-x4qh"
server_key: "013fa66657066c887959cb828a54c7eb"
server_key_iv: "c034b6e2bfe88b185ea8516fedada3cf"
server_key_version: "encryptionkey@0"
I20260809 19:34:31.413048 2340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:31.420091 2340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:31.421707 2340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:31.421808 2340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:31.425495 2340 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:34519
I20260809 19:34:31.425570 2439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:34519 every 8 connection(s)
I20260809 19:34:31.425850 2340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:31.426728 2440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:31.428797 2440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754: Bootstrap starting.
I20260809 19:34:31.429325 2440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:31.429629 2440 log.cc:824] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:31.430274 2440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754: No bootstrap required, opened a new log
I20260809 19:34:31.431843 2440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } }
I20260809 19:34:31.431983 2440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:31.432026 2440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a955a4d0e194deba3924625ab8b3754, State: Initialized, Role: FOLLOWER
I20260809 19:34:31.432154 2440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } }
I20260809 19:34:31.432229 2440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:31.432271 2440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:31.432344 2440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:31.433113 2440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } }
I20260809 19:34:31.433230 2440 leader_election.cc:302] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a955a4d0e194deba3924625ab8b3754; no voters:
I20260809 19:34:31.433436 2440 leader_election.cc:288] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:31.433473 2449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:31.433650 2449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [term 1 LEADER]: Becoming Leader. State: Replica: 4a955a4d0e194deba3924625ab8b3754, State: Running, Role: LEADER
I20260809 19:34:31.433681 2440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:31.433817 2449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } }
I20260809 19:34:31.434259 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a955a4d0e194deba3924625ab8b3754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } } }
I20260809 19:34:31.434302 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2340
I20260809 19:34:31.434289 2453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a955a4d0e194deba3924625ab8b3754. Latest consensus state: current_term: 1 leader_uuid: "4a955a4d0e194deba3924625ab8b3754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a955a4d0e194deba3924625ab8b3754" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 34519 } } }
I20260809 19:34:31.434358 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:31.434368 2453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:31.434422 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:31.434624 31975 external_mini_cluster.cc:1428] Setting key 2b158c4c7d2c46a25373e1a8a07eedc1
I20260809 19:34:31.434624 2463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:31.435134 2463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:31.436648 2463 catalog_manager.cc:1347] Generated new cluster ID: 7c998c62541c44adb699a629cd44aaee
I20260809 19:34:31.436688 2463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:31.444396 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:31.435198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37185 (local address 127.31.57.254:34519)
0809 19:34:31.435475 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:31.435483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:31.439433 (+ 3950us) server_negotiation.cc:407] Connection header received
0809 19:34:31.439588 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:31.439595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:31.439643 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:31.439768 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:31.439952 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:31.441305 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:31.442289 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:31.442915 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:31.443030 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:31.443717 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:31.443720 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:31.443721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:31.443892 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:31.443935 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:31.443948 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:31.443992 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:31.444162 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 19:34:31.444186 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":88,"threads_started":1}
I20260809 19:34:31.445370 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:31.449698 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:31.446695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36809 (local address 127.31.57.254:34519)
0809 19:34:31.446726 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:31.446727 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:31.446857 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:34:31.446909 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:31.446909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:31.446916 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:31.446965 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:31.447061 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:31.447719 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:31.448582 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:31.448832 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:31.448884 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:31.449488 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:31.449488 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:31.449489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:31.449492 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:31.449501 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:31.449504 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:31.449542 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:31.449632 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 19:34:31.449642 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:31.459481 2463 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:31.459928 2463 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:31.464468 2463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a955a4d0e194deba3924625ab8b3754: Generated new TSK 0
I20260809 19:34:31.464614 2463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:31.471738 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2340
W20260809 19:34:31.480237 2314 connection.cc:535] client connection to 127.31.57.254:34519 recv error: Network error: failed to read from TLS socket (remote: 127.31.57.254:34519): Connection reset by peer (error 104)
2026-08-09T19:34:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:31 dist-test-slave-x4qh krb5kdc[2493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:31 dist-test-slave-x4qh krb5kdc[2493](info): set up 2 sockets
Aug 09 19:34:31 dist-test-slave-x4qh krb5kdc[2493](info): commencing operation
krb5kdc: starting...
W20260809 19:34:32.645062 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:32 dist-test-slave-x4qh krb5kdc[2493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:32.754781 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40521
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:40521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:32.830996 2519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:32.831180 2519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:32.831202 2519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:32.832293 2519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:32.832332 2519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:32.832346 2519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:32.832358 2519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:32.832419 2519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:32.833832 2519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:40521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:40521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:32.834079 2519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:32.834331 2519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:32.839102 2525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:32.839143 2519 server_base.cc:1034] running on GCE node
W20260809 19:34:32.839114 2524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:32.839114 2532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:32.839586 2519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:32.839869 2519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:32.841035 2519 hybrid_clock.cc:648] HybridClock initialized: now 1786304072841020 us; error 28 us; skew 500 ppm
Aug 09 19:34:32 dist-test-slave-x4qh krb5kdc[2493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304072, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:32.847668 2519 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:32.848129 2519 webserver.cc:460] Webserver started at http://127.31.57.254:39561/ using document root <none> and password file <none>
I20260809 19:34:32.848372 2519 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:32.848440 2519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:32.848603 2519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:32.849644 2519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "8f6d5a45b887472382623c1f6782c73f"
format_stamp: "Formatted at 2026-08-09 19:34:32 on dist-test-slave-x4qh"
server_key: "428d47dcda36a9f9309017fa22aae190"
server_key_iv: "fb0d545b3c798cee0d7e300c33b2bcdc"
server_key_version: "encryptionkey@0"
I20260809 19:34:32.850006 2519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "8f6d5a45b887472382623c1f6782c73f"
format_stamp: "Formatted at 2026-08-09 19:34:32 on dist-test-slave-x4qh"
server_key: "428d47dcda36a9f9309017fa22aae190"
server_key_iv: "fb0d545b3c798cee0d7e300c33b2bcdc"
server_key_version: "encryptionkey@0"
I20260809 19:34:32.851301 2519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:34:32.852082 2542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:32.852241 2519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:32.852344 2519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "8f6d5a45b887472382623c1f6782c73f"
format_stamp: "Formatted at 2026-08-09 19:34:32 on dist-test-slave-x4qh"
server_key: "428d47dcda36a9f9309017fa22aae190"
server_key_iv: "fb0d545b3c798cee0d7e300c33b2bcdc"
server_key_version: "encryptionkey@0"
I20260809 19:34:32.852422 2519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:32.855497 2519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:32.857189 2519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:32.857344 2519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:32.861363 2519 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:40521
I20260809 19:34:32.861392 2624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:40521 every 8 connection(s)
I20260809 19:34:32.861685 2519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:32.862594 2625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:32.864796 2625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f: Bootstrap starting.
I20260809 19:34:32.865327 2625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:32.865581 2625 log.cc:824] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:32.866182 2625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f: No bootstrap required, opened a new log
I20260809 19:34:32.867795 2625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } }
I20260809 19:34:32.867914 2625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:32.867934 2625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6d5a45b887472382623c1f6782c73f, State: Initialized, Role: FOLLOWER
I20260809 19:34:32.868068 2625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } }
I20260809 19:34:32.868135 2625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:32.868155 2625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:32.868180 2625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:32.868928 2625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } }
I20260809 19:34:32.869062 2625 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6d5a45b887472382623c1f6782c73f; no voters:
I20260809 19:34:32.869199 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2519
I20260809 19:34:32.869233 2625 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:32.869328 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:32.869309 2629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:32.869467 2629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [term 1 LEADER]: Becoming Leader. State: Replica: 8f6d5a45b887472382623c1f6782c73f, State: Running, Role: LEADER
I20260809 19:34:32.869495 2625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:32.869550 31975 external_mini_cluster.cc:1428] Setting key 68a76df6f01c83d31aba3dd00880cbba
I20260809 19:34:32.869547 2629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } }
I20260809 19:34:32.869872 2636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6d5a45b887472382623c1f6782c73f. Latest consensus state: current_term: 1 leader_uuid: "8f6d5a45b887472382623c1f6782c73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } } }
I20260809 19:34:32.869943 2636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:32.870095 2634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6d5a45b887472382623c1f6782c73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6d5a45b887472382623c1f6782c73f" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 40521 } } }
I20260809 19:34:32.870157 2634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:32.870366 2651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:32.871011 2651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:32.872339 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:32.869934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41057 (local address 127.31.57.254:40521)
0809 19:34:32.870138 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:32.870146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:32.870827 (+ 681us) server_negotiation.cc:407] Connection header received
0809 19:34:32.871030 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:32.871038 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:32.871068 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:32.871207 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:32.871819 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:32.871822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:32.871822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:32.871825 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:32.871864 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:32.871877 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:32.871955 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:32.872110 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 19:34:32.872147 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260809 19:34:32.872905 2651 catalog_manager.cc:1347] Generated new cluster ID: 19b0285a0ea4462fb7d3c5bc2e9e9e0d
I20260809 19:34:32.872956 2651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:32.873265 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:32.875756 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:32.874655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45509 (local address 127.31.57.254:40521)
0809 19:34:32.874689 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:32.874691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:32.874818 (+ 127us) server_negotiation.cc:407] Connection header received
0809 19:34:32.874891 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:32.874892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:32.874897 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:32.874947 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:32.875535 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:32.875536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:32.875536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:32.875540 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:32.875552 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:32.875557 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:32.875603 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:32.875653 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:34:32.875667 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:34:32.887674 2651 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:32.887830 2651 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:32.898792 2651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6d5a45b887472382623c1f6782c73f: Generated new TSK 0
I20260809 19:34:32.898972 2651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:32.901245 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2519
2026-08-09T19:34:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:32 dist-test-slave-x4qh krb5kdc[2689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:32 dist-test-slave-x4qh krb5kdc[2689](info): set up 2 sockets
Aug 09 19:34:32 dist-test-slave-x4qh krb5kdc[2689](info): commencing operation
krb5kdc: starting...
W20260809 19:34:34.075687 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:34.191716 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46791
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:46791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:34.275560 2720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:34.275769 2720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:34.275811 2720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:34.276852 2720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:34.276913 2720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:34.276937 2720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:34.276966 2720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:34.276988 2720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:34.278528 2720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:46791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:34.278793 2720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:34.279001 2720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:34.283601 2727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:34.283612 2720 server_base.cc:1034] running on GCE node
W20260809 19:34:34.283609 2732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:34.283718 2726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:34.284071 2720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:34.284439 2720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:34.285600 2720 hybrid_clock.cc:648] HybridClock initialized: now 1786304074285591 us; error 44 us; skew 500 ppm
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304074, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:34.288060 2720 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:34.288420 2720 webserver.cc:460] Webserver started at http://127.31.57.254:33277/ using document root <none> and password file <none>
I20260809 19:34:34.288630 2720 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:34.288669 2720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:34.288766 2720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:34.289657 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "a156bc5e44d042c382ac8ef7429c854b"
format_stamp: "Formatted at 2026-08-09 19:34:34 on dist-test-slave-x4qh"
server_key: "1f38c998d4090413b85af53d1cde6de2"
server_key_iv: "dbd14888cf5031da0422d0de69bc8cc7"
server_key_version: "encryptionkey@0"
I20260809 19:34:34.289925 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "a156bc5e44d042c382ac8ef7429c854b"
format_stamp: "Formatted at 2026-08-09 19:34:34 on dist-test-slave-x4qh"
server_key: "1f38c998d4090413b85af53d1cde6de2"
server_key_iv: "dbd14888cf5031da0422d0de69bc8cc7"
server_key_version: "encryptionkey@0"
I20260809 19:34:34.291150 2720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:34.291890 2746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:34.292054 2720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:34.292135 2720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "a156bc5e44d042c382ac8ef7429c854b"
format_stamp: "Formatted at 2026-08-09 19:34:34 on dist-test-slave-x4qh"
server_key: "1f38c998d4090413b85af53d1cde6de2"
server_key_iv: "dbd14888cf5031da0422d0de69bc8cc7"
server_key_version: "encryptionkey@0"
I20260809 19:34:34.292186 2720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:34.317651 2720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:34.319303 2720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:34.319428 2720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:34.323141 2720 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:46791
I20260809 19:34:34.323194 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:46791 every 8 connection(s)
I20260809 19:34:34.323524 2720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:34.324400 2816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:34.326570 2816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: Bootstrap starting.
I20260809 19:34:34.327104 2816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:34.327450 2816 log.cc:824] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:34.328099 2816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: No bootstrap required, opened a new log
I20260809 19:34:34.329586 2816 raft_consensus.cc:348] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } }
I20260809 19:34:34.329734 2816 raft_consensus.cc:374] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:34.329771 2816 raft_consensus.cc:729] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a156bc5e44d042c382ac8ef7429c854b, State: Initialized, Role: FOLLOWER
I20260809 19:34:34.329944 2816 consensus_queue.cc:260] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } }
I20260809 19:34:34.330027 2816 raft_consensus.cc:388] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:34.330065 2816 raft_consensus.cc:482] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:34.330127 2816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:34.330683 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2720
I20260809 19:34:34.330765 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:34.330993 2816 raft_consensus.cc:504] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } }
I20260809 19:34:34.331079 31975 external_mini_cluster.cc:1428] Setting key 3512e3b2fe232e399270df1736f447c8
I20260809 19:34:34.331107 2816 leader_election.cc:302] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a156bc5e44d042c382ac8ef7429c854b; no voters:
I20260809 19:34:34.331282 2816 leader_election.cc:288] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:34.331307 2822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:34.331424 2822 raft_consensus.cc:686] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [term 1 LEADER]: Becoming Leader. State: Replica: a156bc5e44d042c382ac8ef7429c854b, State: Running, Role: LEADER
I20260809 19:34:34.331557 2816 sys_catalog.cc:564] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:34.331570 2822 consensus_queue.cc:237] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } }
I20260809 19:34:34.331941 2822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a156bc5e44d042c382ac8ef7429c854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } } }
I20260809 19:34:34.332077 2822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:34.332249 2822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a156bc5e44d042c382ac8ef7429c854b. Latest consensus state: current_term: 1 leader_uuid: "a156bc5e44d042c382ac8ef7429c854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a156bc5e44d042c382ac8ef7429c854b" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 46791 } } }
I20260809 19:34:34.332358 2822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:34.333487 2851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:34.333541 2851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:34.333580 2837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:34.333724 2837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:34.335338 2837 catalog_manager.cc:1347] Generated new cluster ID: f5c99dc9f6ad4ca4a7e4f65e1fd85c66
I20260809 19:34:34.335408 2837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:34.341845 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:34.331566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45125 (local address 127.31.57.254:46791)
0809 19:34:34.331744 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:34.331753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:34.331811 (+ 58us) server_negotiation.cc:407] Connection header received
0809 19:34:34.331939 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:34.331946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:34.332001 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:34.332998 (+ 997us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:34.333109 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:34.334590 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:34.335653 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:34.336052 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:34.336176 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:34.338317 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:34.338320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:34.338320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:34.338331 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:34.340378 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:34.340745 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:34.340747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:34.340747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:34.340852 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:34.341054 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:34.341055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:34.341055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:34.341280 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:34.341440 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:34.341505 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:34:34.341638 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260809 19:34:34.342693 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:34.350230 2837 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:34.350630 2837 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:34.353289 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:34.343938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43087 (local address 127.31.57.254:46791)
0809 19:34:34.343974 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:34.343975 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:34.347534 (+ 3559us) server_negotiation.cc:407] Connection header received
0809 19:34:34.347597 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:34.347598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:34.347607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:34.347674 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:34.348157 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:34.348575 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:34.349224 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:34.349410 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:34.349482 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:34.350686 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:34.350688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:34.350688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:34.350695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:34.352161 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:34.352538 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:34.352539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:34.352540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:34.352593 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:34.352797 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:34.352798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:34.352798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:34.352904 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:34.352994 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:34.353120 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 19:34:34.353203 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:34.357511 2837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a156bc5e44d042c382ac8ef7429c854b: Generated new TSK 0
I20260809 19:34:34.357684 2837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:34.359570 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2720
2026-08-09T19:34:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2873](info): set up 2 sockets
Aug 09 19:34:34 dist-test-slave-x4qh krb5kdc[2873](info): commencing operation
krb5kdc: starting...
W20260809 19:34:35.534687 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:35 dist-test-slave-x4qh krb5kdc[2873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:35.644342 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45275
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:45275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:35.719086 2894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:35.719261 2894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:35.719281 2894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:35.720378 2894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:35.720419 2894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:35.720433 2894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:35.720445 2894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:35.720496 2894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:35.721832 2894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:45275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:35.721998 2894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:35.722239 2894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:35.726876 2904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:35.726876 2908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:35.726966 2894 server_base.cc:1034] running on GCE node
W20260809 19:34:35.726897 2901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:35.727408 2894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:35.727710 2894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:35.728875 2894 hybrid_clock.cc:648] HybridClock initialized: now 1786304075728875 us; error 43 us; skew 500 ppm
Aug 09 19:34:35 dist-test-slave-x4qh krb5kdc[2873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304075, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:35.731225 2894 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:35.731685 2894 webserver.cc:460] Webserver started at http://127.31.57.254:46465/ using document root <none> and password file <none>
I20260809 19:34:35.731897 2894 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:35.731963 2894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:35.732118 2894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:35.733171 2894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "6f54b7f090894a2798a1008e39734589"
format_stamp: "Formatted at 2026-08-09 19:34:35 on dist-test-slave-x4qh"
server_key: "2d1067d88530e5e8c957270ca1b84152"
server_key_iv: "48ac05cd07abdbd47bf88dd4c01b8adb"
server_key_version: "encryptionkey@0"
I20260809 19:34:35.733543 2894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "6f54b7f090894a2798a1008e39734589"
format_stamp: "Formatted at 2026-08-09 19:34:35 on dist-test-slave-x4qh"
server_key: "2d1067d88530e5e8c957270ca1b84152"
server_key_iv: "48ac05cd07abdbd47bf88dd4c01b8adb"
server_key_version: "encryptionkey@0"
I20260809 19:34:35.734895 2894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:35.735728 2922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:35.735919 2894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:34:35.736020 2894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "6f54b7f090894a2798a1008e39734589"
format_stamp: "Formatted at 2026-08-09 19:34:35 on dist-test-slave-x4qh"
server_key: "2d1067d88530e5e8c957270ca1b84152"
server_key_iv: "48ac05cd07abdbd47bf88dd4c01b8adb"
server_key_version: "encryptionkey@0"
I20260809 19:34:35.736099 2894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:35.749446 2894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:35.751085 2894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:35.751246 2894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:35.754931 2894 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:45275
I20260809 19:34:35.754997 3008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:45275 every 8 connection(s)
I20260809 19:34:35.755345 2894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:35.756196 3009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:35.758253 3009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589: Bootstrap starting.
I20260809 19:34:35.758580 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 2894
I20260809 19:34:35.758701 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:35.758885 31975 external_mini_cluster.cc:1428] Setting key 073a4df2af1acfc2e37d0d268b926b78
I20260809 19:34:35.758958 3009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:35.759287 3009 log.cc:824] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:35.760025 3009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589: No bootstrap required, opened a new log
I20260809 19:34:35.762072 3009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } }
I20260809 19:34:35.762223 3009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:35.762253 3009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f54b7f090894a2798a1008e39734589, State: Initialized, Role: FOLLOWER
I20260809 19:34:35.762358 3009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } }
I20260809 19:34:35.762446 3009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:35.762499 3009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:35.762545 3009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:35.763108 3009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } }
I20260809 19:34:35.763218 3009 leader_election.cc:302] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f54b7f090894a2798a1008e39734589; no voters:
I20260809 19:34:35.763414 3009 leader_election.cc:288] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:35.763455 3016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:35.763636 3016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [term 1 LEADER]: Becoming Leader. State: Replica: 6f54b7f090894a2798a1008e39734589, State: Running, Role: LEADER
I20260809 19:34:35.763659 3009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:35.763810 3016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } }
I20260809 19:34:35.764143 3018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f54b7f090894a2798a1008e39734589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } } }
I20260809 19:34:35.764227 3018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:35.764578 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:35.759412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33897 (local address 127.31.57.254:45275)
0809 19:34:35.759595 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:35.759603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:35.759634 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:34:35.760137 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:35.760146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:35.760204 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:35.760310 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:35.760566 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:35.761335 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:35.762406 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:35.762879 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:35.762969 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:35.763699 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:35.763704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:35.763704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:35.763916 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:35.763951 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:35.763964 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:35.764023 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:35.764273 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:34:35.764352 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260809 19:34:35.765179 3025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:35.765519 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:35.765568 3017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f54b7f090894a2798a1008e39734589. Latest consensus state: current_term: 1 leader_uuid: "6f54b7f090894a2798a1008e39734589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54b7f090894a2798a1008e39734589" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45275 } } }
I20260809 19:34:35.765638 3017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:35.765990 3025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:35.767485 3025 catalog_manager.cc:1347] Generated new cluster ID: cbe10cec000146ae9486564b03409e3b
I20260809 19:34:35.767539 3025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:35.769711 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:35.766914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35747 (local address 127.31.57.254:45275)
0809 19:34:35.766947 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:35.766948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:35.767082 (+ 134us) server_negotiation.cc:407] Connection header received
0809 19:34:35.767143 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:35.767144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:35.767153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:35.767201 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:35.767301 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:35.767863 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:35.768564 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:35.768767 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:35.768843 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:35.769508 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:35.769509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:35.769510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:35.769514 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:35.769526 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:35.769530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:35.769572 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:35.769623 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:34:35.769642 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:35.783840 3025 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:35.784543 3025 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:35.787190 3025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f54b7f090894a2798a1008e39734589: Generated new TSK 0
I20260809 19:34:35.787348 3025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:35.789157 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 2894
2026-08-09T19:34:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:35 dist-test-slave-x4qh krb5kdc[3054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:35 dist-test-slave-x4qh krb5kdc[3054](info): set up 2 sockets
Aug 09 19:34:35 dist-test-slave-x4qh krb5kdc[3054](info): commencing operation
krb5kdc: starting...
W20260809 19:34:36.954304 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:37 dist-test-slave-x4qh krb5kdc[3054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:37.063194 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39945
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:37.137439 3082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:37.137692 3082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:37.137732 3082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:37.138770 3082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:37.138854 3082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:37.138880 3082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:37.138906 3082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:37.138931 3082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:37.140311 3082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:37.140558 3082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:37.140775 3082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:37.145303 3092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:37.145340 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:37.145401 3082 server_base.cc:1034] running on GCE node
W20260809 19:34:37.145320 3098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:37.145782 3082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:37.146077 3082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:37.147240 3082 hybrid_clock.cc:648] HybridClock initialized: now 1786304077147234 us; error 42 us; skew 500 ppm
Aug 09 19:34:37 dist-test-slave-x4qh krb5kdc[3054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304077, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:37.149549 3082 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:37.149940 3082 webserver.cc:460] Webserver started at http://127.31.57.254:32955/ using document root <none> and password file <none>
I20260809 19:34:37.150143 3082 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:37.150207 3082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:37.150370 3082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:37.151281 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "b0557c4a827d4809ae1c3e425ee2e497"
format_stamp: "Formatted at 2026-08-09 19:34:37 on dist-test-slave-x4qh"
server_key: "e8adcc9f35071b133b2278dc7bc15e4a"
server_key_iv: "c0c447352978a40559b524e79083f3bb"
server_key_version: "encryptionkey@0"
I20260809 19:34:37.151718 3082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "b0557c4a827d4809ae1c3e425ee2e497"
format_stamp: "Formatted at 2026-08-09 19:34:37 on dist-test-slave-x4qh"
server_key: "e8adcc9f35071b133b2278dc7bc15e4a"
server_key_iv: "c0c447352978a40559b524e79083f3bb"
server_key_version: "encryptionkey@0"
I20260809 19:34:37.152974 3082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:37.153663 3111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:37.153913 3082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:37.154008 3082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "b0557c4a827d4809ae1c3e425ee2e497"
format_stamp: "Formatted at 2026-08-09 19:34:37 on dist-test-slave-x4qh"
server_key: "e8adcc9f35071b133b2278dc7bc15e4a"
server_key_iv: "c0c447352978a40559b524e79083f3bb"
server_key_version: "encryptionkey@0"
I20260809 19:34:37.154090 3082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:37.157326 3082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:37.158938 3082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:37.159080 3082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:37.162709 3082 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:39945
I20260809 19:34:37.162748 3192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:39945 every 8 connection(s)
I20260809 19:34:37.163053 3082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:37.163975 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:37.165971 3193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497: Bootstrap starting.
I20260809 19:34:37.166486 3193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:37.166769 3193 log.cc:824] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:37.167266 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3082
I20260809 19:34:37.167411 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:37.167510 3193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497: No bootstrap required, opened a new log
I20260809 19:34:37.167610 31975 external_mini_cluster.cc:1428] Setting key c287e6b51f2d3139110852f651eb7460
I20260809 19:34:37.169720 3193 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } }
I20260809 19:34:37.169874 3193 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:34:37.169916 3193 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0557c4a827d4809ae1c3e425ee2e497, State: Initialized, Role: FOLLOWER
I20260809 19:34:37.170015 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:37.168084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37357 (local address 127.31.57.254:39945)
0809 19:34:37.168265 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:37.168273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:37.168305 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:34:37.168753 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:37.168759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:37.168787 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:37.168887 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:37.169444 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:37.169447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:37.169447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:37.169450 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:37.169481 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:37.169490 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:37.169538 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:37.169614 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:34:37.169639 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":112,"threads_started":1}
I20260809 19:34:37.170025 3193 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } }
I20260809 19:34:37.170182 3193 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:34:37.170219 3193 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:34:37.170264 3193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:37.170902 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:37.170913 3193 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } }
I20260809 19:34:37.170995 3193 leader_election.cc:302] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0557c4a827d4809ae1c3e425ee2e497; no voters:
I20260809 19:34:37.171166 3193 leader_election.cc:288] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:34:37.171223 3202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:37.171391 3193 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:37.171416 3202 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [term 1 LEADER]: Becoming Leader. State: Replica: b0557c4a827d4809ae1c3e425ee2e497, State: Running, Role: LEADER
I20260809 19:34:37.171546 3202 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } }
I20260809 19:34:37.171875 3202 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0557c4a827d4809ae1c3e425ee2e497. Latest consensus state: current_term: 1 leader_uuid: "b0557c4a827d4809ae1c3e425ee2e497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } } }
I20260809 19:34:37.172102 3202 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:37.172019 3203 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0557c4a827d4809ae1c3e425ee2e497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0557c4a827d4809ae1c3e425ee2e497" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39945 } } }
I20260809 19:34:37.172488 3203 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:37.172650 3218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:37.173277 3218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:37.173851 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:37.172532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35749 (local address 127.31.57.254:39945)
0809 19:34:37.172576 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:37.172578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:37.172871 (+ 293us) server_negotiation.cc:407] Connection header received
0809 19:34:37.172968 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:37.172969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:37.172973 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:37.173039 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:37.173662 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:37.173663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:37.173664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:34:37.173668 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:37.173680 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:34:37.173684 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:37.173727 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:37.173778 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:34:37.173790 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:34:37.175020 3218 catalog_manager.cc:1347] Generated new cluster ID: 518854b8336b4ae2b4f062791510ad17
I20260809 19:34:37.175060 3218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:37.195863 3218 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:37.196029 3218 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:37.209007 3218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0557c4a827d4809ae1c3e425ee2e497: Generated new TSK 0
I20260809 19:34:37.209178 3218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:37.219462 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3082
2026-08-09T19:34:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1427 ms)
[----------] 18 tests from AuthTokenIssuingTest (26017 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:37 dist-test-slave-x4qh krb5kdc[3236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:37 dist-test-slave-x4qh krb5kdc[3236](info): set up 2 sockets
Aug 09 19:34:37 dist-test-slave-x4qh krb5kdc[3236](info): commencing operation
krb5kdc: starting...
W20260809 19:34:38.398413 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:38.508229 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45079
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:38.585543 3259 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:38.585759 3259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:38.585798 3259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:38.585860 3259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:38.586896 3259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:38.586953 3259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:38.586977 3259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:38.587003 3259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:38.587029 3259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:38.588385 3259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:45079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:38.588647 3259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:38.588860 3259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:34:38.593696 3259 server_base.cc:1034] running on GCE node
W20260809 19:34:38.593665 3267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.593673 3269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.593664 3273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:38.594120 3259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:38.594401 3259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:38.595561 3259 hybrid_clock.cc:648] HybridClock initialized: now 1786304078595541 us; error 31 us; skew 500 ppm
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:38.602718 3259 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:38.603161 3259 webserver.cc:460] Webserver started at http://127.31.57.254:42587/ using document root <none> and password file <none>
I20260809 19:34:38.603405 3259 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:38.603477 3259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:38.603605 3259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:38.604660 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "903189722af9497fad372fe57e692f77"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "8f0a86ac4db88df4f8cf6c90fc60caa5"
server_key_iv: "fa30e36725f924e0d98a7714175008d7"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.605031 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "903189722af9497fad372fe57e692f77"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "8f0a86ac4db88df4f8cf6c90fc60caa5"
server_key_iv: "fa30e36725f924e0d98a7714175008d7"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.606323 3259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:38.607019 3284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:38.607178 3259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:38.607272 3259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "903189722af9497fad372fe57e692f77"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "8f0a86ac4db88df4f8cf6c90fc60caa5"
server_key_iv: "fa30e36725f924e0d98a7714175008d7"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.607353 3259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:38.628480 3259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:38.630129 3259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:38.630278 3259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:38.634263 3259 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:45079
I20260809 19:34:38.634316 3364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:45079 every 8 connection(s)
I20260809 19:34:38.634641 3259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:38.635600 3365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:38.636950 3365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:38.642477 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3259
I20260809 19:34:38.642603 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:38.642786 31975 external_mini_cluster.cc:1428] Setting key a520ac866792a7ded2e546bad64ae08f
I20260809 19:34:38.645565 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.637485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45079 (local address 127.0.0.1:59526)
0809 19:34:38.637628 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.637667 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.638061 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.638273 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.638275 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.638487 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.638622 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.638626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.639589 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.639590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.640182 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.640183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.640616 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.640617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.640674 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.641169 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.641177 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.642521 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.644242 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.644248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.644250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.644447 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.644701 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.644701 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.644702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.644790 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.645129 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.645134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.645196 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.645294 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.645423 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260809 19:34:38.645887 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.637539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.31.57.254:45079)
0809 19:34:38.637694 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.637698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.637736 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:34:38.638180 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.638182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.638197 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.638261 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.638694 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.639516 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.640248 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.640527 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.640601 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.642619 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.642626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.642626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.642632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.644143 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.644539 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.644542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.644542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.644618 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.644856 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.644857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.644857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.645055 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.645148 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.645711 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.645791 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260809 19:34:38.646497 3365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:38.646811 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.646703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42492)
0809 19:34:38.646746 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.646786 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:34:38.646958 3290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
I20260809 19:34:38.647361 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.647278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42502)
0809 19:34:38.647325 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.647347 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 19:34:38.647480 3365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:45121: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.57.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
I20260809 19:34:38.667475 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:45121
--webserver_interface=127.31.57.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:34:38.696683 3365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } attempt: 1
I20260809 19:34:38.697160 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.697026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42518)
0809 19:34:38.697093 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.697131 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 19:34:38.697504 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.697407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42530)
0809 19:34:38.697454 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.697480 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:34:38.697597 3365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:45121: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
W20260809 19:34:38.747172 3376 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:38.747433 3376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:38.747478 3376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:38.747514 3376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:38.748595 3376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:38.748661 3376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:38.748683 3376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:38.748703 3376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:38.748732 3376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:38.750205 3376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.57.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:38.750481 3376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:38.750712 3376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:38.755383 3385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.755432 3387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.755437 3383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:38.755594 3376 server_base.cc:1034] running on GCE node
I20260809 19:34:38.755919 3376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:38.756222 3376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:38.757391 3376 hybrid_clock.cc:648] HybridClock initialized: now 1786304078757379 us; error 44 us; skew 500 ppm
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:38.759840 3376 init.cc:377] Logged in from keytab as kudu/127.31.57.253@KRBTEST.COM (short username kudu)
I20260809 19:34:38.760232 3376 webserver.cc:460] Webserver started at http://127.31.57.253:43879/ using document root <none> and password file <none>
I20260809 19:34:38.760469 3376 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:38.760536 3376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:38.760679 3376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:38.761693 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data/instance:
uuid: "83062272f8054839aae3eeb3ba309ec9"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "d64f490bfa9c2af8dda8a8aed39ccb86"
server_key_iv: "6148eaadd72f349eea67abd3153a45bf"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.762072 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal/instance:
uuid: "83062272f8054839aae3eeb3ba309ec9"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "d64f490bfa9c2af8dda8a8aed39ccb86"
server_key_iv: "6148eaadd72f349eea67abd3153a45bf"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.763320 3376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:34:38.764043 3405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:38.764210 3376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:38.764318 3376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "83062272f8054839aae3eeb3ba309ec9"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "d64f490bfa9c2af8dda8a8aed39ccb86"
server_key_iv: "6148eaadd72f349eea67abd3153a45bf"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.764415 3376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:38.765733 3365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } attempt: 2
I20260809 19:34:38.766085 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.765977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42538)
0809 19:34:38.766029 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.766060 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:34:38.766407 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.766321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42542)
0809 19:34:38.766363 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.766385 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:34:38.766489 3365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:45121: Network error: Client connection negotiation failed: client connection to 127.31.57.253:45121: connect: Connection refused (error 111)
I20260809 19:34:38.773990 3376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:38.775728 3376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:38.775884 3376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:38.779760 3376 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.253:45121
I20260809 19:34:38.779824 3483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.253:45121 every 8 connection(s)
I20260809 19:34:38.780205 3376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
I20260809 19:34:38.781080 3484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:38.782377 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3376
I20260809 19:34:38.782467 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-1/wal/instance
I20260809 19:34:38.782653 31975 external_mini_cluster.cc:1428] Setting key fc656321d0b600d2f7828284f9b6e1ac
I20260809 19:34:38.782523 3484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.57.252@KRBTEST.COM; defaulting to no policy
I20260809 19:34:38.792809 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.783311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.31.57.254:45079)
0809 19:34:38.783398 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.783401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.783662 (+ 261us) server_negotiation.cc:407] Connection header received
0809 19:34:38.784383 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.784385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.784395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.784482 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.785145 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.785874 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.786868 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.787219 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.787312 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.789770 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.789771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.789772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.789779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.791498 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.791890 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.791891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.791891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.791954 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.792219 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.792220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.792220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.792376 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.792453 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.792681 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.792747 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 19:34:38.792946 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.783285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45079 (local address 127.0.0.1:59530)
0809 19:34:38.783520 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.783566 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.784281 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.784581 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.784585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.784864 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.785047 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.785051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.785967 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.785968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.786769 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.786771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.787333 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.787334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.787523 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.788010 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.788017 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.789673 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.791578 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.791585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.791586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.791811 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.792024 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.792024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.792025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.792165 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.792448 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.792453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.792570 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.792647 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.792752 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":134,"threads_started":1}
I20260809 19:34:38.793540 3484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.252@KRBTEST.COM" created.
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:38.800974 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.793761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42556)
0809 19:34:38.793810 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.793820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.793860 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.794128 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.794129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.794261 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.794307 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.794308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.795342 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.795343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.795846 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.795847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.796325 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.796326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.796390 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.796791 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.796795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.798068 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.799803 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.799805 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.799805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.800036 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.800241 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.800242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.800242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.800316 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.800597 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.800598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.800649 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.800725 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.800826 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:34:38.800992 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.793844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.31.57.253:45121)
0809 19:34:38.794003 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.794006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.794022 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:34:38.794038 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.794039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.794048 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.794129 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.794383 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.795234 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.795941 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.796221 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.796319 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.798150 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.798158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.798158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.798165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.799721 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.800119 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.800122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.800122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.800173 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.800379 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.800380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.800380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.800510 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.800585 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.800759 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.800890 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260809 19:34:38.801601 3484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:38.801952 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.801825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47762)
0809 19:34:38.801874 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.801922 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:34:38.802109 3407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
I20260809 19:34:38.802549 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.802442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47772)
0809 19:34:38.802493 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.802523 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:34:38.802683 3484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:40899: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
I20260809 19:34:38.809720 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:40899
--webserver_interface=127.31.57.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:34:38.851917 3484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } attempt: 1
I20260809 19:34:38.852396 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.852230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47774)
0809 19:34:38.852308 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.852354 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 19:34:38.852705 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.852635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47788)
0809 19:34:38.852662 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.852686 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 19:34:38.852803 3484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:40899: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
I20260809 19:34:38.857630 3365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } attempt: 3
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:38.864565 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.857940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42568)
0809 19:34:38.857991 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.858004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.858065 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.858198 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.858199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.858393 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.858477 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.858479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.859126 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.859127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.859935 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.859939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.860300 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.860301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.860380 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.860780 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.860786 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.862088 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.863664 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.863665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.863665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.863819 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.864013 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.864014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.864014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.864062 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.864324 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.864325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.864372 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.864433 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.864504 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:34:38.864614 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.857988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.31.57.253:45121)
0809 19:34:38.858048 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.858050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.858059 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:38.858129 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.858129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.858138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.858181 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.858550 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.859046 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.860019 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.860223 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.860283 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.862165 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.862166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.862167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.862173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.863591 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.863888 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.863889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.863890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.863949 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.864117 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.864118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.864118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.864239 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.864330 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.864456 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.864532 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:34:38.865320 3365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:38.865653 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.865552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47804)
0809 19:34:38.865600 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.865628 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:34:38.866019 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.865929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47812)
0809 19:34:38.865983 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.866006 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 19:34:38.866135 3365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:40899: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
W20260809 19:34:38.890604 3498 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:38.890777 3498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:38.890800 3498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:38.890818 3498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:38.891911 3498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:38.891948 3498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:38.891961 3498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:38.891973 3498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:38.891983 3498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:38.893319 3498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:45079,127.31.57.253:45121,127.31.57.252:40899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:40899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.57.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:38.893509 3498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:38.893661 3498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:34:38.897253 3365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } attempt: 1
I20260809 19:34:38.897831 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.897606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47826)
0809 19:34:38.897743 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.897783 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 19:34:38.898260 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.898169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47838)
0809 19:34:38.898209 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.898231 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 19:34:38.898389 3365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:40899: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
W20260809 19:34:38.898362 3505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.898391 3510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:38.898391 3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:38.898937 3498 server_base.cc:1034] running on GCE node
I20260809 19:34:38.899164 3498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:38.899482 3498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:38.900635 3498 hybrid_clock.cc:648] HybridClock initialized: now 1786304078900626 us; error 37 us; skew 500 ppm
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:38.902834 3498 init.cc:377] Logged in from keytab as kudu/127.31.57.252@KRBTEST.COM (short username kudu)
I20260809 19:34:38.903256 3498 webserver.cc:460] Webserver started at http://127.31.57.252:37559/ using document root <none> and password file <none>
I20260809 19:34:38.903532 3498 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:38.903595 3498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:38.903726 3498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:38.904738 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data/instance:
uuid: "c20f3812cbc0466f91c6af74c106e7c8"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "4a13910123352781a34bc64f7d58ad5b"
server_key_iv: "099ab3cc983d4a6f35f82d7394a7cea2"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.905099 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal/instance:
uuid: "c20f3812cbc0466f91c6af74c106e7c8"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "4a13910123352781a34bc64f7d58ad5b"
server_key_iv: "099ab3cc983d4a6f35f82d7394a7cea2"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.906365 3498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:38.907056 3523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:38.907239 3498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:38.907331 3498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "c20f3812cbc0466f91c6af74c106e7c8"
format_stamp: "Formatted at 2026-08-09 19:34:38 on dist-test-slave-x4qh"
server_key: "4a13910123352781a34bc64f7d58ad5b"
server_key_iv: "099ab3cc983d4a6f35f82d7394a7cea2"
server_key_version: "encryptionkey@0"
I20260809 19:34:38.907423 3498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:38.920944 3484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } attempt: 2
I20260809 19:34:38.921402 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.921282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47850)
0809 19:34:38.921340 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.921375 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:34:38.921680 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.921617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47854)
0809 19:34:38.921643 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.921665 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 19:34:38.921798 3484 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:40899: Network error: Client connection negotiation failed: client connection to 127.31.57.252:40899: connect: Connection refused (error 111)
I20260809 19:34:38.925666 3498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:38.927356 3498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:38.927553 3498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:38.931536 3498 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.252:40899
I20260809 19:34:38.931615 3612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.252:40899 every 8 connection(s)
I20260809 19:34:38.931949 3498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
I20260809 19:34:38.932907 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:38.934366 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3498
I20260809 19:34:38.934484 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303811347120-31975-0/minicluster-data/master-2/wal/instance
I20260809 19:34:38.934418 3614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:38.934689 31975 external_mini_cluster.cc:1428] Setting key 6039bb2b091f0dab8961ec6557728771
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:38.943499 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.935197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.31.57.254:45079)
0809 19:34:38.935276 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.935278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.935526 (+ 248us) server_negotiation.cc:407] Connection header received
0809 19:34:38.936027 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.936028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.936039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.936103 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.936618 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.937090 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.937944 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.938132 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.938203 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.940469 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.940471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.940471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.940479 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.942049 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.942527 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.942528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.942529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.942604 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.942860 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.942861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.942861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.942976 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.943047 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.943325 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.943416 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 19:34:38.943681 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.935200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:45079 (local address 127.0.0.1:59546)
0809 19:34:38.935397 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.935439 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.935919 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.936133 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.936135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.936388 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.936525 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.936529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.937175 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.937176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.937850 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.937852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.938228 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.938230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.938364 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.938866 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.938875 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.940384 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.942153 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.942161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.942163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.942447 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.942680 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.942681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.942681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.942784 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.943050 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.943057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.943187 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.943309 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.943465 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260809 19:34:38.943703 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.935425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.31.57.252:40899)
0809 19:34:38.935550 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.935554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.935591 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:34:38.935943 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.935953 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.935972 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.936084 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.936715 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.937468 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.938151 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.938454 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.938536 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.940897 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.940905 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.940906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.940912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.942445 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.942840 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.942844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.942844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.942902 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.943104 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.943105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.943105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.943239 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.943444 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.943525 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.943597 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260809 19:34:38.944407 3614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:38.951208 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.944636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:45121 (local address 127.0.0.1:42582)
0809 19:34:38.944677 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.944686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.944719 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.944851 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.944852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.945000 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.945036 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.945038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.945788 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.945788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.946545 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.946548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.946929 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.946930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.946969 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.947311 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.947317 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.948547 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.950215 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.950216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.950217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.950383 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.950594 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.950595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.950595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.950656 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.950903 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.950904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.950944 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.951026 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.951114 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:34:38.951223 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.944709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.31.57.253:45121)
0809 19:34:38.944748 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.944749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.944757 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:38.944781 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.944782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.944791 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.944850 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.945105 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.945710 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.946630 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.946862 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.946921 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.948632 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.948633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.948633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.948639 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.950138 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.950467 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.950469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.950469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.950531 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.950726 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.950727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.950728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.950838 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.950909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.951065 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.951132 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:38.951716 3614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:38.951763 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.944709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.31.57.254:45079)
0809 19:34:38.944741 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.944743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.944753 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:34:38.944774 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.944775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.944791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.944850 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.945354 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.945968 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.946802 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.947039 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.947115 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.949230 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.949231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.949231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.949238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.950586 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.950915 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.950916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.950916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.950968 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.951173 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.951174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.951175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.951312 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.951506 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.951613 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.951683 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:38.958664 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.951957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47870)
0809 19:34:38.951989 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.951999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.952034 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.952374 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.952375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.952504 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.952530 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.952531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.953224 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.953224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.953673 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.953674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.954064 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.954077 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.954120 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.954520 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.954524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.955988 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.957684 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.957685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.957686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.957918 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.958107 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.958108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.958108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.958168 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.958384 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.958385 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.958430 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.958528 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.958612 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 19:34:38.958868 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.952148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.31.57.252:40899)
0809 19:34:38.952201 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.952203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.952210 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:38.952228 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.952229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.952238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.952306 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.952601 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.953152 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.953767 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.953983 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.954062 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.956066 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.956067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.956068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.956073 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.957599 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.957991 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.957992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.957992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.958040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.958231 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.958232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.958232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.958321 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.958368 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.958552 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.958668 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":35}
I20260809 19:34:38.959703 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.952329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.31.57.253:45121)
0809 19:34:38.952361 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.952363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.952681 (+ 318us) server_negotiation.cc:407] Connection header received
0809 19:34:38.952701 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.952701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.952710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.952765 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.953168 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.953624 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.954238 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.954469 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.954534 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.956923 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.956925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.956925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.956932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.958465 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.958820 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.958822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.958822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.958876 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.959197 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.959198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.959198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.959329 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.959522 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.959586 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.959636 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:38.960415 3614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: Bootstrap starting.
I20260809 19:34:38.960968 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:38.961284 3614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:38.961553 3614 log.cc:824] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:38.962320 3614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: No bootstrap required, opened a new log
I20260809 19:34:38.962188 3325 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "903189722af9497fad372fe57e692f77"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59552
I20260809 19:34:38.963791 3614 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.963925 3614 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20f3812cbc0466f91c6af74c106e7c8, State: Initialized, Role: FOLLOWER
I20260809 19:34:38.964038 3614 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.964309 3614 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:38.964301 3622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.964424 3622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:34:38.966141 3644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:38.966197 3644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:38.966476 3498 master_runner.cc:186] Error getting master registration for 127.31.57.254:45079: OK, instance_id { permanent_uuid: "903189722af9497fad372fe57e692f77" instance_seqno: 1786304078633162 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:34:38.968456 3498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:34:38.973513 3365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } attempt: 2
Aug 09 19:34:38 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:38.980394 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.973796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.31.57.252:40899)
0809 19:34:38.973828 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.973830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.973839 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:38.973881 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.973881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.973890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.973945 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.974207 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.974888 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.975913 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.976198 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.976253 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.977921 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.977922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.977923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.977930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.979246 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.979600 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.979601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.979601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.979658 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.979851 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.979852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.979852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.979970 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.980063 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.980212 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.980279 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:34:38.980410 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.973738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47886)
0809 19:34:38.973793 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:38.973802 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:38.973836 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:38.973950 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:38.973950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:38.974101 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:38.974141 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.974142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.974970 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.974971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.975807 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:38.975810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.976268 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.976269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:38.976326 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.976649 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:38.976654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:38.977837 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:38.979333 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.979333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.979334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.979529 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.979726 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:38.979727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:38.979727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:38.979780 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:38.980057 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:38.980058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:38.980100 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:34:38.980188 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:34:38.980287 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:34:38.983093 3365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77: Bootstrap starting.
I20260809 19:34:38.983886 3365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:38.984212 3365 log.cc:824] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:38.985038 3365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77: No bootstrap required, opened a new log
I20260809 19:34:38.986325 3365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.986475 3365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903189722af9497fad372fe57e692f77, State: Initialized, Role: FOLLOWER
I20260809 19:34:38.986543 3365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.986841 3365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:38.986827 3647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.986918 3325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:34:38.987007 3647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:38.987071 3325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:38.988014 3325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.988557 3561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "903189722af9497fad372fe57e692f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c20f3812cbc0466f91c6af74c106e7c8"
I20260809 19:34:38.988677 3561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:38.989578 3561 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903189722af9497fad372fe57e692f77 in term 1.
I20260809 19:34:38.989764 3325 leader_election.cc:288] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [CANDIDATE]: Term 1 election: Requested vote from peers 83062272f8054839aae3eeb3ba309ec9 (127.31.57.253:45121), c20f3812cbc0466f91c6af74c106e7c8 (127.31.57.252:40899)
I20260809 19:34:38.989823 3288 leader_election.cc:302] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903189722af9497fad372fe57e692f77, c20f3812cbc0466f91c6af74c106e7c8; no voters:
I20260809 19:34:38.990278 3647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:38.990420 3647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [term 1 LEADER]: Becoming Leader. State: Replica: 903189722af9497fad372fe57e692f77, State: Running, Role: LEADER
I20260809 19:34:38.990494 3647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:38.990869 3647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 903189722af9497fad372fe57e692f77. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.990963 3647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:38.989594 3259 master_runner.cc:186] Error getting master registration for 127.31.57.253:45121: OK, instance_id { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" instance_seqno: 1786304078778633 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:34:38.992920 3259 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:34:38.993084 3679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:38.993559 3679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:38.995543 3561 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [term 1 FOLLOWER]: Refusing update from remote peer 903189722af9497fad372fe57e692f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:38.995904 3668 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:38.997265 3622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 903189722af9497fad372fe57e692f77. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.997349 3622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:38.998587 3647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.998672 3647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:38.998741 3622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:38.998839 3622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:38.998914 3684 mvcc.cc:205] Tried to move back new op lower bound from 7316701507561840640 to 7316701507545632768. Current Snapshot: MvccSnapshot[applied={T|T < 7316701507561840640}]
I20260809 19:34:38.999068 3685 mvcc.cc:205] Tried to move back new op lower bound from 7316701507561840640 to 7316701507545632768. Current Snapshot: MvccSnapshot[applied={T|T < 7316701507561840640}]
I20260809 19:34:38.999266 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.991088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.31.57.254:45079)
0809 19:34:38.991120 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.991122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.991129 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:38.991147 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.991147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.991155 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.991246 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:38.991984 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.992540 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.993524 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:38.993811 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:38.993857 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:38.996112 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:38.996113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:38.996114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:38.996120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:38.997763 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.998134 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.998135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.998136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.998191 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:38.998411 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:38.998412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:38.998412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:38.998548 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:38.998638 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:38.999108 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 19:34:38.999176 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:39.006609 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.991520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.31.57.252:40899)
0809 19:34:38.991552 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.991554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.999343 (+ 7789us) server_negotiation.cc:407] Connection header received
0809 19:34:38.999504 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.999505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.999514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.999541 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.000259 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.000917 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.001789 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.002038 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.002096 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:39.003387 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:39.003388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:39.003389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:39.003396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:39.005160 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.005667 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.005669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.005669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.005728 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.006046 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.006047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.006048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.006176 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:39.006269 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:39.006442 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:34:39.006506 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:39.006779 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:38.991860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.31.57.253:45121)
0809 19:34:38.991906 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:38.991908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:38.999514 (+ 7606us) server_negotiation.cc:407] Connection header received
0809 19:34:38.999617 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:38.999619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:38.999630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:38.999661 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.000384 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.001016 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.002135 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.002770 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.002818 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:39.004189 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:39.004191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:39.004191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:39.004217 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:39.005642 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.006026 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.006027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.006027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.006080 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.006272 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.006272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.006273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.006384 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:39.006444 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:39.006624 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:34:39.006696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:34:39.012931 3484 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } attempt: 3
I20260809 19:34:39.013659 3679 catalog_manager.cc:1347] Generated new cluster ID: 630b217f18984550a9111fff23abe51b
I20260809 19:34:39.013713 3679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:34:39 dist-test-slave-x4qh krb5kdc[3236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304078, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:39.020922 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:39.013169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:40899 (local address 127.0.0.1:47908)
0809 19:34:39.013209 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:39.013219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:39.013258 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:39.013391 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:39.013393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:39.013521 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:39.013578 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:39.013580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.014415 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.014416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:39.015172 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:39.015175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.015942 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.015943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:39.015999 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:39.016408 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:39.016413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:39.017914 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:39.019840 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:39.019841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:39.019842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:39.020056 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:39.020262 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:39.020262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:39.020263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:39.020325 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:39.020624 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:39.020624 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:39.020679 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:34:39.020738 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:34:39.020835 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 19:34:39.021220 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:39.013211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.31.57.252:40899)
0809 19:34:39.013257 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.013259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.013267 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:39.013317 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.013318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.013326 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.013380 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.013666 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.014328 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.015249 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:39.015520 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:39.015577 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:39.018003 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:39.018004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:39.018005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:39.018011 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:39.019750 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.020138 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.020139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.020139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.020194 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:39.020399 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:39.020400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:39.020401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:39.020517 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:39.020611 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:39.020775 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 19:34:39.020883 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:34:39.022714 3484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: Bootstrap starting.
I20260809 19:34:39.023427 3484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:39.023711 3484 log.cc:824] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:39.024541 3484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: No bootstrap required, opened a new log
I20260809 19:34:39.026618 3484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:39.026739 3484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83062272f8054839aae3eeb3ba309ec9, State: Initialized, Role: FOLLOWER
I20260809 19:34:39.026818 3484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } }
I20260809 19:34:39.027114 3484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:39.027170 3445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:39.028227 3445 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [term 1 FOLLOWER]: Refusing update from remote peer 903189722af9497fad372fe57e692f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:39.028429 3446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "903189722af9497fad372fe57e692f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "83062272f8054839aae3eeb3ba309ec9"
I20260809 19:34:39.028616 3690 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:39.029197 3446 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903189722af9497fad372fe57e692f77 in term 1.
I20260809 19:34:39.027074 3696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:39.029584 3696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:39.030565 3696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 903189722af9497fad372fe57e692f77. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:39.030807 3696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:39.032929 3696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:39.033048 3696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:34:39.031364 3714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:39.033176 3714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:39.039449 3679 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:39.040136 3679 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:34:39.040516 3690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "903189722af9497fad372fe57e692f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903189722af9497fad372fe57e692f77" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 45079 } } peers { permanent_uuid: "83062272f8054839aae3eeb3ba309ec9" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 45121 } } peers { permanent_uuid: "c20f3812cbc0466f91c6af74c106e7c8" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 40899 } } }
I20260809 19:34:39.040608 3690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:39.052644 3679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 903189722af9497fad372fe57e692f77: Generated new TSK 0
I20260809 19:34:39.052894 3679 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:34:39.085750 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.084358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.31.57.253:45121)
0809 19:34:39.084401 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.084402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.085076 (+ 674us) server_negotiation.cc:407] Connection header received
0809 19:34:39.085184 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.085185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.085196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.085226 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.085689 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42606: BlockingRecv error: recv got EOF from 127.0.0.1:42606 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:34:39.086519 3619 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.084795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.31.57.252:40899)
0809 19:34:39.084834 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.084836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.086001 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 19:34:39.086096 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.086097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.086107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.086135 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.086464 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47912: BlockingRecv error: recv got EOF from 127.0.0.1:47912 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:34:39.087513 3368 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.086832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.31.57.254:45079)
0809 19:34:39.086868 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.086870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.086981 (+ 111us) server_negotiation.cc:407] Connection header received
0809 19:34:39.087068 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.087069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.087079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.087106 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.087454 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59562: BlockingRecv error: recv got EOF from 127.0.0.1:59562 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:34:39.092378 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.091662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.31.57.253:45121)
0809 19:34:39.091695 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.091696 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.091869 (+ 173us) server_negotiation.cc:407] Connection header received
0809 19:34:39.091983 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.091984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.091999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.092027 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.092338 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42620: BlockingRecv error: recv got EOF from 127.0.0.1:42620 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:34:39.095314 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.094539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.31.57.253:45121)
0809 19:34:39.094568 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.094569 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.094826 (+ 257us) server_negotiation.cc:407] Connection header received
0809 19:34:39.094923 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.094924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.094933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.094962 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.095264 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42630: BlockingRecv error: recv got EOF from 127.0.0.1:42630 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:34:39.099081 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.098317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.31.57.253:45121)
0809 19:34:39.098349 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.098362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.098577 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:34:39.098665 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.098666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.098675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.098702 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.099044 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42646: BlockingRecv error: recv got EOF from 127.0.0.1:42646 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:34:39.104843 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.104031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.31.57.253:45121)
0809 19:34:39.104059 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.104061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.104299 (+ 238us) server_negotiation.cc:407] Connection header received
0809 19:34:39.104408 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.104409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.104418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.104444 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.104804 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42650: BlockingRecv error: recv got EOF from 127.0.0.1:42650 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:34:39.114494 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.113782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.31.57.253:45121)
0809 19:34:39.113813 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.113815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.114033 (+ 218us) server_negotiation.cc:407] Connection header received
0809 19:34:39.114118 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.114119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.114127 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.114153 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.114454 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42662: BlockingRecv error: recv got EOF from 127.0.0.1:42662 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:34:39.132557 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.131611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.31.57.253:45121)
0809 19:34:39.131648 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.131650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.131911 (+ 261us) server_negotiation.cc:407] Connection header received
0809 19:34:39.132011 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.132013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.132023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.132052 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.132509 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42666: BlockingRecv error: recv got EOF from 127.0.0.1:42666 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:34:39.166632 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.165897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.31.57.253:45121)
0809 19:34:39.165945 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.165947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.166107 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:34:39.166181 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.166182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.166194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.166245 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.166580 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42668: BlockingRecv error: recv got EOF from 127.0.0.1:42668 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 19:34:39.232463 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.231819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.31.57.253:45121)
0809 19:34:39.231891 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.231893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.231950 (+ 57us) server_negotiation.cc:407] Connection header received
0809 19:34:39.232000 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.232001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.232010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.232060 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.232381 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42680: BlockingRecv error: recv got EOF from 127.0.0.1:42680 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 19:34:39.362216 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.361528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.31.57.253:45121)
0809 19:34:39.361616 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.361618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.361683 (+ 65us) server_negotiation.cc:407] Connection header received
0809 19:34:39.361711 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.361712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.361722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.361790 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.362121 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42692: BlockingRecv error: recv got EOF from 127.0.0.1:42692 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 19:34:39.620076 3492 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:39.619489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.31.57.253:45121)
0809 19:34:39.619553 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:39.619558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:39.619570 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:34:39.619616 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:39.619617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:39.619627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:39.619686 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:39.619998 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42700: BlockingRecv error: recv got EOF from 127.0.0.1:42700 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:34:39.966601 3644 catalog_manager.cc:1259] Loaded cluster ID: 630b217f18984550a9111fff23abe51b
I20260809 19:34:39.966665 3644 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: loading cluster ID for follower catalog manager: success
I20260809 19:34:39.967458 3644 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: acquiring CA information for follower catalog manager: success
I20260809 19:34:39.967689 3644 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c20f3812cbc0466f91c6af74c106e7c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:34:40.033778 3714 catalog_manager.cc:1259] Loaded cluster ID: 630b217f18984550a9111fff23abe51b
I20260809 19:34:40.033878 3714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: loading cluster ID for follower catalog manager: success
I20260809 19:34:40.034729 3714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: acquiring CA information for follower catalog manager: success
I20260809 19:34:40.035045 3714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83062272f8054839aae3eeb3ba309ec9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:34:40.135541 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:40.133219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.31.57.253:45121)
0809 19:34:40.133322 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:40.133324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:40.133336 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:34:40.133414 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:40.133415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:40.133461 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:40.133531 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:40.133673 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.134120 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.134785 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.134978 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.135026 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:40.135179 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:40.135284 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:40.135341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:40.135415 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 19:34:40.135437 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
I20260809 19:34:40.139329 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:40.137367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.31.57.252:40899)
0809 19:34:40.137461 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:40.137463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:40.137471 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:40.137505 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:40.137505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:40.137537 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:40.137596 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:40.137703 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.138128 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.138721 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.138895 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.138939 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:40.139052 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:40.139130 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:40.139191 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:40.139222 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:34:40.139236 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":49,"threads_started":1}
I20260809 19:34:40.143898 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:40.140836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.31.57.253:45121)
0809 19:34:40.140993 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:40.140996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:40.141114 (+ 118us) server_negotiation.cc:407] Connection header received
0809 19:34:40.141187 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:40.141188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:40.141196 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:40.141246 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:40.141386 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.142004 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.143108 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.143427 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.143483 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:40.143653 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:40.143682 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:40.143741 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:40.143818 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:34:40.143838 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260809 19:34:40.144167 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:40.140888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.31.57.254:45079)
0809 19:34:40.141000 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:40.141002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:40.141305 (+ 303us) server_negotiation.cc:407] Connection header received
0809 19:34:40.141356 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:40.141357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:40.141402 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:40.141512 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:40.141622 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.142299 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.142984 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.143282 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.143346 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:40.143575 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:40.143690 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:40.143749 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:40.144067 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 19:34:40.144089 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260809 19:34:40.144487 3839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:40.141000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.31.57.252:40899)
0809 19:34:40.141034 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:40.141036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:40.141134 (+ 98us) server_negotiation.cc:407] Connection header received
0809 19:34:40.141159 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:40.141159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:40.141170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:40.141220 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:40.141325 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.142042 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.143466 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:40.143762 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:40.143825 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:40.144075 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:40.144106 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:40.144133 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:40.144393 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 19:34:40.144420 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:34:40.145821 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3259
I20260809 19:34:40.163400 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3376
I20260809 19:34:40.171957 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3498
2026-08-09T19:34:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2952 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:34:40 dist-test-slave-x4qh krb5kdc[3859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:34:40 dist-test-slave-x4qh krb5kdc[3859](info): set up 2 sockets
Aug 09 19:34:40 dist-test-slave-x4qh krb5kdc[3859](info): commencing operation
krb5kdc: starting...
W20260809 19:34:41.363432 31975 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/test-user.keytab.
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:34:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.57.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
Entry for principal kudu/127.31.57.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab.
I20260809 19:34:41.474220 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39701
--webserver_interface=127.31.57.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:34:41.554456 3876 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:41.554690 3876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:41.554721 3876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:41.554764 3876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:41.555948 3876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:41.556052 3876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:41.556082 3876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:41.556110 3876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:41.556133 3876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:41.557621 3876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.254.keytab
--principal=kudu/127.31.57.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.254:39701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.57.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:41.557880 3876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:41.558120 3876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:41.562922 3886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.562901 3888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.562913 3890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:41.563145 3876 server_base.cc:1034] running on GCE node
I20260809 19:34:41.563405 3876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:41.563675 3876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:41.564836 3876 hybrid_clock.cc:648] HybridClock initialized: now 1786304081564822 us; error 27 us; skew 500 ppm
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:41.567147 3876 init.cc:377] Logged in from keytab as kudu/127.31.57.254@KRBTEST.COM (short username kudu)
I20260809 19:34:41.567610 3876 webserver.cc:460] Webserver started at http://127.31.57.254:42345/ using document root <none> and password file <none>
I20260809 19:34:41.567848 3876 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:41.567911 3876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:41.568071 3876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:41.569067 3876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data/instance:
uuid: "acf87cbd312241aeb1ede966a26cac20"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "52b9d226249127e35857c4a6b0139e6d"
server_key_iv: "8d1c992302715bf04cb880c6fc648035"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.569411 3876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance:
uuid: "acf87cbd312241aeb1ede966a26cac20"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "52b9d226249127e35857c4a6b0139e6d"
server_key_iv: "8d1c992302715bf04cb880c6fc648035"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.570688 3876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:34:41.571338 3903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:41.571506 3876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:34:41.571622 3876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal
uuid: "acf87cbd312241aeb1ede966a26cac20"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "52b9d226249127e35857c4a6b0139e6d"
server_key_iv: "8d1c992302715bf04cb880c6fc648035"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.571703 3876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:41.578207 3876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:41.579900 3876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:41.580052 3876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:41.583923 3876 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.254:39701
I20260809 19:34:41.583954 3987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.254:39701 every 8 connection(s)
I20260809 19:34:41.584301 3876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/data/info.pb
I20260809 19:34:41.585253 3988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:41.586637 3988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.588577 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 3876
I20260809 19:34:41.588687 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-0/wal/instance
I20260809 19:34:41.588877 31975 external_mini_cluster.cc:1428] Setting key 7893f80c0ebb0dc9727dee8c9a39b447
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:41.596901 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.587226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39701 (local address 127.0.0.1:45134)
0809 19:34:41.587434 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.587479 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.587883 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.588149 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.588151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.588369 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.588516 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.588519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.589648 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.589649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.590290 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.590292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.590798 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.590800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.590867 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.591451 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.591459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.593146 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.595352 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.595361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.595401 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.595615 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.595878 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.595878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.595879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.595956 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.596435 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.596442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.596512 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.596579 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.596691 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260809 19:34:41.596922 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.587428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.31.57.254:39701)
0809 19:34:41.587559 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.587563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.587589 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:34:41.588005 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.588006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.588021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.588113 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.588621 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.589551 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.590385 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.590674 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.590798 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.593252 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.593262 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.593262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.593270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.595233 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.595698 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.595702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.595702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.595803 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.596050 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.596051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.596052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.596353 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.596438 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.596664 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.596751 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.31.57.253@KRBTEST.COM; defaulting to no policy
I20260809 19:34:41.598709 3988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.599107 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.598936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49034)
0809 19:34:41.599001 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.599074 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:34:41.599273 3910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
I20260809 19:34:41.599690 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.599606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49044)
0809 19:34:41.599648 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.599673 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:34:41.599857 3988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:39533: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
Entry for principal kudu/127.31.57.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab.
I20260809 19:34:41.613869 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:39533
--webserver_interface=127.31.57.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:34:41.649179 3988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } attempt: 1
I20260809 19:34:41.649642 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.649503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49054)
0809 19:34:41.649581 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.649618 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 19:34:41.649956 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.649866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49062)
0809 19:34:41.649909 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.649934 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:34:41.650043 3988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:39533: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
W20260809 19:34:41.691473 4003 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:41.691690 4003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:41.691725 4003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:41.691762 4003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:41.692833 4003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:41.692896 4003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:41.692921 4003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:41.692948 4003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:41.692974 4003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:41.694389 4003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.253.keytab
--principal=kudu/127.31.57.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.253:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.57.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:41.694644 4003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:41.694873 4003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:34:41.699599 4003 server_base.cc:1034] running on GCE node
W20260809 19:34:41.699589 4018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.699579 4013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.699582 4012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:41.700006 4003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:41.700323 4003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:41.701493 4003 hybrid_clock.cc:648] HybridClock initialized: now 1786304081701485 us; error 37 us; skew 500 ppm
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:41.703858 4003 init.cc:377] Logged in from keytab as kudu/127.31.57.253@KRBTEST.COM (short username kudu)
I20260809 19:34:41.704315 4003 webserver.cc:460] Webserver started at http://127.31.57.253:41119/ using document root <none> and password file <none>
I20260809 19:34:41.704543 4003 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:41.704608 4003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:41.704789 4003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:41.705775 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data/instance:
uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "6b390de47a39281ca8440b1f3e65d0c0"
server_key_iv: "947de8fe369ca3907893549f76c37abf"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.706152 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal/instance:
uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "6b390de47a39281ca8440b1f3e65d0c0"
server_key_iv: "947de8fe369ca3907893549f76c37abf"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.707424 4003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:34:41.708112 4027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:41.708282 4003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:34:41.708375 4003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal
uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "6b390de47a39281ca8440b1f3e65d0c0"
server_key_iv: "947de8fe369ca3907893549f76c37abf"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.708463 4003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:41.718261 3988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } attempt: 2
I20260809 19:34:41.718641 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.718528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49064)
0809 19:34:41.718583 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.718617 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:34:41.718932 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.718843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49078)
0809 19:34:41.718887 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.718910 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:34:41.719018 3988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.253:39533: Network error: Client connection negotiation failed: client connection to 127.31.57.253:39533: connect: Connection refused (error 111)
I20260809 19:34:41.737396 4003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:41.739311 4003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:41.739513 4003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:41.743567 4003 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.253:39533
I20260809 19:34:41.743604 4110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.253:39533 every 8 connection(s)
I20260809 19:34:41.744001 4003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/data/info.pb
I20260809 19:34:41.744983 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:41.746497 4114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.748513 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4003
I20260809 19:34:41.748633 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-1/wal/instance
I20260809 19:34:41.748803 31975 external_mini_cluster.cc:1428] Setting key 411327ce50130236826e2135144ffaea
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:41.755535 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.747194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.31.57.254:39701)
0809 19:34:41.747262 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.747264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.747488 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:34:41.747923 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.747925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.747935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.747984 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.748653 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.749213 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.750190 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.750403 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.750486 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.752643 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.752644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.752644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.752651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.754059 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.754473 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.754475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.754475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.754537 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.754804 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.754805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.754806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.754926 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.755024 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.755342 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.755449 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:34:41.755686 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.747176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39701 (local address 127.0.0.1:45150)
0809 19:34:41.747357 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.747406 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.747833 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.748056 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.748058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.748302 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.748533 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.748537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.749329 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.749330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.750099 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.750101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.750515 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.750517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.750631 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.751137 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.751145 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.752540 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.754160 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.754168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.754170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.754392 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.754612 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.754613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.754613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.754720 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.755016 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.755023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.755159 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.755284 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.755465 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260809 19:34:41.756278 4114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.57.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.57.252@KRBTEST.COM" created.
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:41.763517 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.756543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49090)
0809 19:34:41.756601 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.756614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.756661 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.756884 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.756885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.757067 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.757140 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.757142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.757928 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.757929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.758434 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.758435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.759007 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.759008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.759068 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.759490 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.759495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.760652 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.762433 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.762434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.762434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.762586 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.762836 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.762836 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.762837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.762889 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.763182 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.763183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.763218 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.763283 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.763411 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 19:34:41.763617 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.756581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.31.57.253:39533)
0809 19:34:41.756728 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.756732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.756750 (+ 18us) server_negotiation.cc:407] Connection header received
0809 19:34:41.756785 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.756787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.756797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.756860 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.757213 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.757832 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.758504 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.758911 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.758999 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.760731 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.760740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.760740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.760748 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.762321 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.762676 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.762678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.762678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.762738 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.762958 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.762959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.762959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.763109 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.763201 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.763308 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.763494 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 19:34:41.764099 4114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.764456 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.764351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35600)
0809 19:34:41.764399 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.764438 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:34:41.764616 4029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
I20260809 19:34:41.765008 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.764923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35602)
0809 19:34:41.764970 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.764993 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 19:34:41.765128 4114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:37401: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
Entry for principal kudu/127.31.57.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab.
I20260809 19:34:41.773496 31975 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrXpU3x/build/release/bin/kudu
/tmp/dist-test-taskrXpU3x/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:37401
--webserver_interface=127.31.57.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.57.212:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:34:41.810333 3988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } attempt: 3
I20260809 19:34:41.814337 4114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } attempt: 1
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:41.814779 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.814643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35606)
0809 19:34:41.814705 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.814742 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:34:41.815114 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.815038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35622)
0809 19:34:41.815062 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.815086 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 19:34:41.815438 4114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:37401: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
I20260809 19:34:41.817456 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.810720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49096)
0809 19:34:41.810786 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.810795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.810855 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.811025 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.811027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.811185 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.811247 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.811249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.811981 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.811982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.812528 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.812529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.812940 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.812941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.813043 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.813491 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.813502 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.814784 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.816559 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.816560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.816560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.816725 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.816956 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.816957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.816957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.817009 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.817243 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.817244 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.817282 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.817331 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.817400 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 19:34:41.817513 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.810780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.31.57.253:39533)
0809 19:34:41.810844 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.810847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.810857 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:34:41.810922 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.810923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.810934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.811003 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.811338 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.811889 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.812616 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.812867 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.812943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.814909 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.814911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.814911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.814918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.816474 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.816833 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.816834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.816834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.816892 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.817068 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.817069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.817069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.817186 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.817247 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.817367 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.817439 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:34:41.817816 3988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.818094 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.818017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35632)
0809 19:34:41.818046 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.818072 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 19:34:41.818410 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.818331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35642)
0809 19:34:41.818374 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.818396 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 19:34:41.818513 3988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:37401: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
I20260809 19:34:41.849679 3988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } attempt: 1
I20260809 19:34:41.850167 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.850035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35646)
0809 19:34:41.850104 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.850139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 19:34:41.850584 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.850497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35650)
0809 19:34:41.850541 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.850566 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:34:41.850695 3988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:37401: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
W20260809 19:34:41.852780 4125 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:34:41.852984 4125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:34:41.853017 4125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:34:41.853055 4125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:34:41.854079 4125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:34:41.854137 4125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:34:41.854161 4125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:34:41.854187 4125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:34:41.854213 4125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:34:41.855706 4125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.57.212:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.57.254:39701,127.31.57.253:39533,127.31.57.252:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/krb5kdc/kudu_127.31.57.252.keytab
--principal=kudu/127.31.57.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.57.252:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.57.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:34:41.855970 4125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:34:41.856184 4125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:34:41.861052 4132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.861076 4131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:34:41.861222 4134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:34:41.861233 4125 server_base.cc:1034] running on GCE node
I20260809 19:34:41.861531 4125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:34:41.861830 4125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:34:41.862972 4125 hybrid_clock.cc:648] HybridClock initialized: now 1786304081862949 us; error 39 us; skew 500 ppm
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:34:41.865425 4125 init.cc:377] Logged in from keytab as kudu/127.31.57.252@KRBTEST.COM (short username kudu)
I20260809 19:34:41.865883 4125 webserver.cc:460] Webserver started at http://127.31.57.252:34877/ using document root <none> and password file <none>
I20260809 19:34:41.866110 4125 fs_manager.cc:359] Metadata directory not provided
I20260809 19:34:41.866151 4125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:34:41.866298 4125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:34:41.867220 4125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data/instance:
uuid: "9eb0277691a642098efb04cec2445526"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "243731193fe9208a7314a46f50e3c10b"
server_key_iv: "bd0d8ccc1e8082ef1f67f99ede9ed997"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.867589 4125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal/instance:
uuid: "9eb0277691a642098efb04cec2445526"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "243731193fe9208a7314a46f50e3c10b"
server_key_iv: "bd0d8ccc1e8082ef1f67f99ede9ed997"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.869899 4125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 19:34:41.870705 4142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:34:41.870904 4125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:34:41.870976 4125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data,/tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal
uuid: "9eb0277691a642098efb04cec2445526"
format_stamp: "Formatted at 2026-08-09 19:34:41 on dist-test-slave-x4qh"
server_key: "243731193fe9208a7314a46f50e3c10b"
server_key_iv: "bd0d8ccc1e8082ef1f67f99ede9ed997"
server_key_version: "encryptionkey@0"
I20260809 19:34:41.871026 4125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:34:41.883584 4114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } attempt: 2
I20260809 19:34:41.883975 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.883849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35666)
0809 19:34:41.883907 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.883943 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:34:41.884348 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.884252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35670)
0809 19:34:41.884299 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.884322 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:34:41.884485 4114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.57.252:37401: Network error: Client connection negotiation failed: client connection to 127.31.57.252:37401: connect: Connection refused (error 111)
I20260809 19:34:41.886988 4125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:34:41.889262 4125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:34:41.889434 4125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:34:41.893374 4125 rpc_server.cc:307] RPC server started. Bound to: 127.31.57.252:37401
I20260809 19:34:41.893427 4221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.57.252:37401 every 8 connection(s)
I20260809 19:34:41.893783 4125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/data/info.pb
I20260809 19:34:41.894660 4222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:34:41.896148 4222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.898149 31975 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrXpU3x/build/release/bin/kudu as pid 4125
I20260809 19:34:41.898264 31975 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrXpU3x/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303811347120-31975-0/minicluster-data/master-2/wal/instance
I20260809 19:34:41.898466 31975 external_mini_cluster.cc:1428] Setting key 0e1d1b3315c30aa0593e8e457ac9eb21
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:41.906256 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.896805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.31.57.254:39701)
0809 19:34:41.896865 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.896867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.897067 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:34:41.897549 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.897550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.897562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.897612 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.898208 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.898999 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.900497 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.900814 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.900899 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.903285 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.903286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.903287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.903294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.904909 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.905303 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.905304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.905304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.905365 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.905620 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.905621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.905621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.905732 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.905807 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.906060 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.906143 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:34:41.906518 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.896758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.254:39701 (local address 127.0.0.1:45154)
0809 19:34:41.896944 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.896974 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.897439 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.897677 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.897680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.897894 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.898117 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.898120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.899112 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.899113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.900372 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.900375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.900939 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.900940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.901062 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.901562 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.901569 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.903194 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.904995 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.905003 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.905005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.905217 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.905438 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.905439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.905439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.905541 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.905803 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.905810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.905930 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.906036 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.906259 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260809 19:34:41.907032 4222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.907770 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.898864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.31.57.252:37401)
0809 19:34:41.899054 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.899057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.899072 (+ 15us) server_negotiation.cc:407] Connection header received
0809 19:34:41.899131 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.899132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.899147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.899259 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.900439 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.901316 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.902125 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.902397 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.902450 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.904772 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.904778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.904778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.904785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.906322 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.906734 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.906737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.906737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.906809 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.907032 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.907033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.907033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.907254 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.907426 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.907547 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.907644 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.254@KRBTEST.COM
I20260809 19:34:41.913830 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.907302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.253:39533 (local address 127.0.0.1:49102)
0809 19:34:41.907400 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.907410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.907455 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.907579 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.907580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.907720 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.907746 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.907747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.908491 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.908492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.909037 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.909039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.909377 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.909378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.909434 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.909808 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.909812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.911005 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.912707 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.912708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.912709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.912947 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.913174 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.913175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.913175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.913238 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.913507 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.913508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.913546 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.913643 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.913751 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 19:34:41.913843 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.907345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.31.57.253:39533)
0809 19:34:41.907406 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.907409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.907441 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:34:41.907513 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.907514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.907523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.907576 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.907833 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.908397 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.909117 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.909308 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.909358 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.911081 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.911083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.911083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.911089 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.912629 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.913021 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.913023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.913023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.913100 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.913316 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.913317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.913317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.913436 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.913503 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.913669 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.913756 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:34:41.914315 4222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:34:41.915917 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.908463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.31.57.254:39701)
0809 19:34:41.908496 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.908498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.908507 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:34:41.908527 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.908528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.908535 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.908594 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.908985 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.909534 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.910172 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.910356 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.910423 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.913192 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.913193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.913193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.913199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.914764 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.915138 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.915139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.915139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.915191 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.915440 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.915441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.915442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.915549 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.915713 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.915749 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.915826 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.252@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.57.253@KRBTEST.COM
I20260809 19:34:41.920837 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.914561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35702)
0809 19:34:41.914642 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.914652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.914687 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.914980 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.914981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.915082 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.915106 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.915107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.915781 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.915782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.916524 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.916526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.916941 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.916942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.916986 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.917280 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.917283 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.918231 (+ 948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.919861 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.919862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.919862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.920019 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.920225 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.920226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.920227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.920282 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.920465 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.920466 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.920507 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.920576 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.920678 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 19:34:41.920878 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.914581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.31.57.252:37401)
0809 19:34:41.914876 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.914878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.914885 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:41.914903 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.914904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.914913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.914967 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.915199 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.915697 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.916610 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.916872 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.916942 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.918331 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.918332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.918332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.918338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.919788 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.920093 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.920094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.920094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.920155 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.920338 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.920338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.920338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.920409 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.920446 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.920607 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.920750 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":277}
I20260809 19:34:41.922451 4222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: Bootstrap starting.
I20260809 19:34:41.923453 4222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:41.923620 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.916538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.31.57.253:39533)
0809 19:34:41.916588 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.916589 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.916630 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:34:41.916680 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.916681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.916689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.916734 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.917333 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.917699 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.918387 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.918654 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.918715 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.921112 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.921113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.921114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.921119 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.922539 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.922909 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.922910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.922910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.922957 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.923148 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.923148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.923148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.923230 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.923332 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.923467 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.923548 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:34:41.923774 4222 log.cc:824] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:41.924612 4222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: No bootstrap required, opened a new log
I20260809 19:34:41.925033 31975 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:34:41.925817 3988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } attempt: 2
I20260809 19:34:41.926273 4222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.926401 4222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb0277691a642098efb04cec2445526, State: Initialized, Role: FOLLOWER
I20260809 19:34:41.926481 4222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.926025 3949 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "acf87cbd312241aeb1ede966a26cac20"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45156
I20260809 19:34:41.926791 4222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:41.926748 4230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.926844 4230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:34:41.928802 4250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:41.928876 4250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.254@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:41.928799 4125 master_runner.cc:186] Error getting master registration for 127.31.57.254:39701: OK, instance_id { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" instance_seqno: 1786304081582730 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:34:41.931034 4125 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:34:41.933698 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.927112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.31.57.252:37401)
0809 19:34:41.927143 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.927144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.927172 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:34:41.927245 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.927246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.927254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.927300 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.927618 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.928213 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.929076 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.929343 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.929409 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.931200 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.931201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.931201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.931207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.932538 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.932939 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.932940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.932941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.932996 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.933210 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.933211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.933211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.933352 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.933411 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.933557 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.933622 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 19:34:41.933705 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.927076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35718)
0809 19:34:41.927122 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.927133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.927186 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.927318 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.927319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.927507 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.927546 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.927548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.928306 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.928307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.928994 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.928996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.929409 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.929410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.929463 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.929825 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.929831 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.931117 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.932620 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.932621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.932640 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.932857 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.933068 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.933069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.933069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.933134 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.933422 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.933422 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.933464 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.933532 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.933622 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 19:34:41.935148 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: Bootstrap starting.
I20260809 19:34:41.935916 3988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:41.936234 3988 log.cc:824] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:41.937104 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: No bootstrap required, opened a new log
I20260809 19:34:41.938408 3988 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.938541 3988 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf87cbd312241aeb1ede966a26cac20, State: Initialized, Role: FOLLOWER
I20260809 19:34:41.938622 3988 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.938928 3988 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:41.938979 3949 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:34:41.938882 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.939055 3949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:41.939128 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:41.939877 3949 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.940119 3949 leader_election.cc:288] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [CANDIDATE]: Term 1 election: Requested vote from peers 7abaa7a8e8754b0a8e7acc46d3dd74c0 (127.31.57.253:39533), 9eb0277691a642098efb04cec2445526 (127.31.57.252:37401)
I20260809 19:34:41.940362 4186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "acf87cbd312241aeb1ede966a26cac20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9eb0277691a642098efb04cec2445526"
I20260809 19:34:41.940510 4186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:41.941452 4186 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf87cbd312241aeb1ede966a26cac20 in term 1.
I20260809 19:34:41.941679 3908 leader_election.cc:302] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb0277691a642098efb04cec2445526, acf87cbd312241aeb1ede966a26cac20; no voters:
I20260809 19:34:41.941808 4253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:34:41.941954 4253 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [term 1 LEADER]: Becoming Leader. State: Replica: acf87cbd312241aeb1ede966a26cac20, State: Running, Role: LEADER
I20260809 19:34:41.942159 4253 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.942657 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf87cbd312241aeb1ede966a26cac20. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.942728 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: This master's current role is: LEADER
W20260809 19:34:41.943581 4286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:34:41.943630 4286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:34:41.943714 4295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:34:41.943832 4295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:34:41.945144 4186 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [term 1 FOLLOWER]: Refusing update from remote peer acf87cbd312241aeb1ede966a26cac20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:41.945461 4288 consensus_queue.cc:1035] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:41.944155 3876 master_runner.cc:186] Error getting master registration for 127.31.57.253:39533: OK, instance_id { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" instance_seqno: 1786304081742463 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:34:41.946460 3876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:34:41.946573 4230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf87cbd312241aeb1ede966a26cac20. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.946653 4230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:41.948655 4288 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.948797 4288 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:41.949158 4230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.949225 4230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:41.949383 4295 catalog_manager.cc:1347] Generated new cluster ID: f600dedd30074b54b40f6d382645e8c0
I20260809 19:34:41.949411 4298 mvcc.cc:205] Tried to move back new op lower bound from 7316701519645179904 to 7316701519635767296. Current Snapshot: MvccSnapshot[applied={T|T < 7316701519645179904 or (T in {7316701519645179904})}]
I20260809 19:34:41.949427 4295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:34:41.949572 4299 mvcc.cc:205] Tried to move back new op lower bound from 7316701519645179904 to 7316701519635767296. Current Snapshot: MvccSnapshot[applied={T|T < 7316701519645179904}]
I20260809 19:34:41.954422 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.941177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.31.57.252:37401)
0809 19:34:41.941207 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.941209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.947654 (+ 6445us) server_negotiation.cc:407] Connection header received
0809 19:34:41.947750 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.947752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.947763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.947825 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.948602 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.949220 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.950181 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.950452 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.950488 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.951878 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.951879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.951879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.951885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.953261 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.953626 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.953627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.953628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.953674 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.953867 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.953868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.953868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.953993 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.954066 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.954262 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.954344 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:41.961644 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.941153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.31.57.254:39701)
0809 19:34:41.941203 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.941205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.951532 (+ 10327us) server_negotiation.cc:407] Connection header received
0809 19:34:41.951599 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.951600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.951609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.951655 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.952931 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.953517 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.954382 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.954628 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.954657 (+ 29us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.958575 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.958576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.958577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.958582 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.960548 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.960892 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.960893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.960893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.960940 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.961105 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.961106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.961106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.961208 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.961279 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.961487 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.961554 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:41.961843 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.941550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.31.57.253:39533)
0809 19:34:41.941589 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.941591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.955107 (+ 13516us) server_negotiation.cc:407] Connection header received
0809 19:34:41.955197 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.955198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.955207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.955235 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.956069 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.956773 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.957668 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.957949 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.957985 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.959164 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.959166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.959166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.959172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.960698 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.961053 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.961055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.961055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.961103 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.961293 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.961294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.961294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.961425 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.961488 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.961679 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.961753 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:34:41.974949 4295 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:34:41.975647 4114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } attempt: 3
I20260809 19:34:41.976017 4295 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 19:34:41 dist-test-slave-x4qh krb5kdc[3859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786304081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.57.253@KRBTEST.COM for kudu/127.31.57.252@KRBTEST.COM
I20260809 19:34:41.983551 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.975878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.57.252:37401 (local address 127.0.0.1:35732)
0809 19:34:41.975925 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:34:41.975935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:34:41.975976 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:34:41.976237 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:34:41.976238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:34:41.976398 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:34:41.976434 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.976435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.977244 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.977245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.978034 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:34:41.978037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.978563 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.978564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:34:41.978630 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.979017 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:34:41.979022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:34:41.980563 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:34:41.982342 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.982343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.982344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.982546 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.982776 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:34:41.982777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:34:41.982777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:34:41.982897 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:34:41.983215 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:34:41.983216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:34:41.983270 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 19:34:41.983345 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:34:41.983454 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:34:41.984017 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:41.976051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.31.57.252:37401)
0809 19:34:41.976093 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:41.976095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:41.976100 (+ 5us) server_negotiation.cc:407] Connection header received
0809 19:34:41.976117 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:41.976117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:41.976124 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:41.976164 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:41.976510 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.977134 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.978143 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:41.978445 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:41.978499 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:41.980671 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:34:41.980672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:34:41.980672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:34:41.980678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:34:41.982232 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.982635 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.982636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.982636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.982699 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:34:41.982978 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:34:41.982979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:34:41.982979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:34:41.983114 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:34:41.983170 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:41.983863 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 19:34:41.983935 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:41.985814 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: Bootstrap starting.
I20260809 19:34:41.986522 4114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: Neither blocks nor log segments found. Creating new log.
I20260809 19:34:41.986822 4114 log.cc:824] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:34:41.988401 4295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20: Generated new TSK 0
I20260809 19:34:41.988557 4295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:34:41.989261 4114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: No bootstrap required, opened a new log
I20260809 19:34:41.991205 4114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.991350 4114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abaa7a8e8754b0a8e7acc46d3dd74c0, State: Initialized, Role: FOLLOWER
I20260809 19:34:41.991460 4114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } }
I20260809 19:34:41.991757 4114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:34:41.991843 4073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:34:41.992632 4073 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [term 1 FOLLOWER]: Refusing update from remote peer acf87cbd312241aeb1ede966a26cac20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:34:41.992800 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "acf87cbd312241aeb1ede966a26cac20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0"
I20260809 19:34:41.993489 4075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf87cbd312241aeb1ede966a26cac20 in term 1.
I20260809 19:34:41.994022 4253 consensus_queue.cc:1035] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:34:41.991755 4308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.996047 4308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:41.999500 4326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf87cbd312241aeb1ede966a26cac20. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.999591 4326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:41.999696 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:41.999737 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:34:42.002502 4322 catalog_manager.cc:1259] Loaded cluster ID: f600dedd30074b54b40f6d382645e8c0
I20260809 19:34:42.002595 4322 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: loading cluster ID for follower catalog manager: success
I20260809 19:34:42.003756 4322 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: acquiring CA information for follower catalog manager: success
I20260809 19:34:42.004132 4322 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7abaa7a8e8754b0a8e7acc46d3dd74c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 19:34:42.019644 4121 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.018115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.31.57.253:39533)
0809 19:34:42.018163 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.018166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.019031 (+ 865us) server_negotiation.cc:407] Connection header received
0809 19:34:42.019125 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.019126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.019136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.019163 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.019592 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49136: BlockingRecv error: recv got EOF from 127.0.0.1:49136 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 19:34:42.020433 3998 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.018500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.31.57.254:39701)
0809 19:34:42.018535 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.018537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.019960 (+ 1423us) server_negotiation.cc:407] Connection header received
0809 19:34:42.020046 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.020047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.020055 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.020081 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.020370 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45170: BlockingRecv error: recv got EOF from 127.0.0.1:45170 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:34:42.021163 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.018799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.31.57.252:37401)
0809 19:34:42.018831 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.018832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.020718 (+ 1886us) server_negotiation.cc:407] Connection header received
0809 19:34:42.020804 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.020805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.020814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.020840 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.021114 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35736: BlockingRecv error: recv got EOF from 127.0.0.1:35736 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:42.027431 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.024076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.31.57.253:39533)
0809 19:34:42.024107 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.024109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.024306 (+ 197us) server_negotiation.cc:407] Connection header received
0809 19:34:42.024426 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.024426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.024456 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.024496 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.024688 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.025431 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.026558 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.026802 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.026840 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.027095 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.027192 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.027232 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.027340 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.027357 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:34:42.032331 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.031608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.31.57.252:37401)
0809 19:34:42.031641 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.031642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.031852 (+ 210us) server_negotiation.cc:407] Connection header received
0809 19:34:42.031960 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.031961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.031975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.032001 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.032280 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35748: BlockingRecv error: recv got EOF from 127.0.0.1:35748 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:42.037530 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.034610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.31.57.253:39533)
0809 19:34:42.034638 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.034640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.034847 (+ 207us) server_negotiation.cc:407] Connection header received
0809 19:34:42.034930 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.034931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.034938 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.034969 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.035153 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.035834 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.036831 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.037092 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.037127 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.037311 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.037339 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.037365 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.037460 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.037475 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:34:42.040279 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.039617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.31.57.252:37401)
0809 19:34:42.039647 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.039648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.039850 (+ 202us) server_negotiation.cc:407] Connection header received
0809 19:34:42.039930 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.039931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.039941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.039966 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.040240 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35750: BlockingRecv error: recv got EOF from 127.0.0.1:35750 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:34:42.046332 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.043321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.31.57.253:39533)
0809 19:34:42.043350 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.043352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.043647 (+ 295us) server_negotiation.cc:407] Connection header received
0809 19:34:42.043731 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.043732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.043740 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.043764 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.043932 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.044572 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.045564 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.045815 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.045855 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.046049 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.046078 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.046104 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.046213 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.046230 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:34:42.049301 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.048604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.31.57.252:37401)
0809 19:34:42.048636 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.048638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.048826 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:34:42.048923 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.048924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.048933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.048961 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.049248 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35756: BlockingRecv error: recv got EOF from 127.0.0.1:35756 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:42.057492 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.054417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.31.57.253:39533)
0809 19:34:42.054451 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.054453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.054691 (+ 238us) server_negotiation.cc:407] Connection header received
0809 19:34:42.054788 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.054789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.054799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.054828 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.055011 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.055709 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.056739 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.056996 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.057040 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.057237 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.057265 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.057308 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.057417 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.057434 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:34:42.060467 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.059769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.31.57.252:37401)
0809 19:34:42.059802 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.059804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.059988 (+ 184us) server_negotiation.cc:407] Connection header received
0809 19:34:42.060080 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.060081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.060090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.060118 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.060428 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35760: BlockingRecv error: recv got EOF from 127.0.0.1:35760 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:42.072660 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.069558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.31.57.253:39533)
0809 19:34:42.069591 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.069592 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.069822 (+ 230us) server_negotiation.cc:407] Connection header received
0809 19:34:42.069932 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.069933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.069942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.069969 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.070131 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.070843 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.071940 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.072184 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.072225 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.072429 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.072459 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.072485 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.072584 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.072601 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:34:42.075740 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.074983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.31.57.252:37401)
0809 19:34:42.075033 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.075035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.075203 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:34:42.075305 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.075306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.075316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.075345 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.075697 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35764: BlockingRecv error: recv got EOF from 127.0.0.1:35764 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:34:42.096464 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.093120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.31.57.253:39533)
0809 19:34:42.093160 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.093162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.093462 (+ 300us) server_negotiation.cc:407] Connection header received
0809 19:34:42.093566 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.093567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.093578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.093607 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.093797 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.094528 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.095625 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.095908 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.095951 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.096178 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.096211 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.096240 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.096369 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.096390 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:34:42.099699 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.098905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.31.57.252:37401)
0809 19:34:42.098941 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.098943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.099113 (+ 170us) server_negotiation.cc:407] Connection header received
0809 19:34:42.099197 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.099198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.099207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.099245 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.099650 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35770: BlockingRecv error: recv got EOF from 127.0.0.1:35770 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:42.135612 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.133037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.31.57.253:39533)
0809 19:34:42.133092 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.133094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.133196 (+ 102us) server_negotiation.cc:407] Connection header received
0809 19:34:42.133231 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.133232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.133242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.133315 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.133446 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.134135 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.134813 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.135105 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.135179 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.135309 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.135358 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.135451 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.135499 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.135521 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 19:34:42.137933 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.137403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.31.57.252:37401)
0809 19:34:42.137458 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.137460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.137491 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:34:42.137545 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.137546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.137555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.137612 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.137880 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35782: BlockingRecv error: recv got EOF from 127.0.0.1:35782 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:34:42.205049 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.203017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.31.57.253:39533)
0809 19:34:42.203097 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.203099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.203174 (+ 75us) server_negotiation.cc:407] Connection header received
0809 19:34:42.203219 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.203220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.203228 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.203295 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.203425 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.203856 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.204501 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.204686 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.204734 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.204857 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.204882 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.204918 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.204977 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.204992 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 19:34:42.207290 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.206716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.31.57.252:37401)
0809 19:34:42.206797 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.206799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.206844 (+ 45us) server_negotiation.cc:407] Connection header received
0809 19:34:42.206885 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.206886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.206894 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.206937 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.207232 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35796: BlockingRecv error: recv got EOF from 127.0.0.1:35796 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 19:34:42.338864 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.336454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.31.57.253:39533)
0809 19:34:42.336545 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.336547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.336639 (+ 92us) server_negotiation.cc:407] Connection header received
0809 19:34:42.336737 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.336739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.336748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.336828 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.336970 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.337423 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.338154 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.338417 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.338489 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.338630 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.338668 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.338713 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.338762 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.338782 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 19:34:42.341176 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.340615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.31.57.252:37401)
0809 19:34:42.340670 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.340671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.340715 (+ 44us) server_negotiation.cc:407] Connection header received
0809 19:34:42.340757 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.340758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.340766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.340830 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.341119 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35810: BlockingRecv error: recv got EOF from 127.0.0.1:35810 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:34:42.434607 4253 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "acf87cbd312241aeb1ede966a26cac20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf87cbd312241aeb1ede966a26cac20" member_type: VOTER last_known_addr { host: "127.31.57.254" port: 39701 } } peers { permanent_uuid: "7abaa7a8e8754b0a8e7acc46d3dd74c0" member_type: VOTER last_known_addr { host: "127.31.57.253" port: 39533 } } peers { permanent_uuid: "9eb0277691a642098efb04cec2445526" member_type: VOTER last_known_addr { host: "127.31.57.252" port: 37401 } } }
I20260809 19:34:42.434768 4253 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf87cbd312241aeb1ede966a26cac20 [sys.catalog]: This master's current role is: LEADER
I20260809 19:34:42.600907 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:42.598641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.31.57.253:39533)
0809 19:34:42.598722 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.598725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.598733 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:34:42.598762 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.598763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.598774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.598844 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:42.599011 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.599468 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.600213 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:42.600411 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:42.600488 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:42.600625 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:42.600669 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:42.600733 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:42.600802 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:34:42.600822 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 19:34:42.603284 4228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:34:42.602741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.31.57.252:37401)
0809 19:34:42.602783 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:42.602785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:42.602807 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:34:42.602833 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:42.602833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:42.602842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:42.602886 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:34:42.603215 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35820: BlockingRecv error: recv got EOF from 127.0.0.1:35820 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:34:42.929315 4250 catalog_manager.cc:1259] Loaded cluster ID: f600dedd30074b54b40f6d382645e8c0
I20260809 19:34:42.929383 4250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: loading cluster ID for follower catalog manager: success
I20260809 19:34:42.930084 4250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: acquiring CA information for follower catalog manager: success
I20260809 19:34:42.930318 4250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9eb0277691a642098efb04cec2445526: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:34:43.118619 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:43.116352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.31.57.253:39533)
0809 19:34:43.116486 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:43.116487 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:43.116520 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:34:43.116592 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:43.116593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:43.116603 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:43.116659 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:43.116777 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.117237 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.117852 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.118052 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.118141 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:43.118295 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:43.118327 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:43.118375 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:43.118479 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 19:34:43.118501 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 19:34:43.122442 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:43.120351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.31.57.252:37401)
0809 19:34:43.120449 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:43.120451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:43.120490 (+ 39us) server_negotiation.cc:407] Connection header received
0809 19:34:43.120511 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:43.120512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:43.120560 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:43.120621 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:43.120725 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.121186 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.121824 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.122001 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.122047 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:43.122170 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:43.122275 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:43.122335 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:43.122369 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:34:43.122385 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":59,"threads_started":1}
I20260809 19:34:43.126994 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:43.124063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.31.57.253:39533)
0809 19:34:43.124097 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:43.124098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:43.124246 (+ 148us) server_negotiation.cc:407] Connection header received
0809 19:34:43.124349 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:43.124350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:43.124360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:43.124406 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:43.124567 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.125003 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126164 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.126446 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126498 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:43.126745 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:43.126773 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:43.126819 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:43.126906 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:34:43.126926 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:34:43.126924 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:43.124110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.31.57.252:37401)
0809 19:34:43.124152 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:43.124154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:43.124161 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:43.124230 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:43.124231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:43.124239 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:43.124313 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:43.124439 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.125126 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126130 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.126498 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126558 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:43.126700 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:43.126731 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:43.126778 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:43.126831 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 19:34:43.126847 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:34:43.127106 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:34:43.124163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.31.57.254:39701)
0809 19:34:43.124264 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 19:34:43.124266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:34:43.124273 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:34:43.124326 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:34:43.124327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:34:43.124404 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:34:43.124491 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:34:43.124616 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.125346 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126082 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:34:43.126416 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:34:43.126476 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:34:43.126652 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:34:43.126743 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:34:43.126982 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 19:34:43.126999 (+ 17us) server_negotiation.cc:299] Negotiation successful
0809 19:34:43.127023 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":52,"threads_started":1}
I20260809 19:34:43.128836 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 3876
I20260809 19:34:43.146147 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4003
I20260809 19:34:43.155454 31975 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrXpU3x/build/release/bin/kudu with pid 4125
2026-08-09T19:34:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2981 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5934 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271818 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST